masdeseiscaracteres/gcolab_spark_install.md

Created September 30, 2020 12:52

Star (0) You must be signed in to star a gist
Fork (0) You must be signed in to fork a gist

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/masdeseiscaracteres/af9c915393a929e0c535a2f00af34abd.js"></script>
Save masdeseiscaracteres/af9c915393a929e0c535a2f00af34abd to your computer and use it in GitHub Desktop.

Instalar Spark en Google Colab

Raw

!apt-get install openjdk-8-jdk-headless -qq > /dev/null
!wget -q https://www-us.apache.org/dist/spark/spark-2.4.7/spark-2.4.7-bin-hadoop2.7.tgz
!tar xf spark-2.4.7-bin-hadoop2.7.tgz
!pip install -q findspark

import os
os.environ["JAVA_HOME"] = "/usr/lib/jvm/java-8-openjdk-amd64"
os.environ["SPARK_HOME"] = "/content/spark-2.4.7-bin-hadoop2.7"
import findspark
findspark.init()

from pyspark.sql import SparkSession
spark = SparkSession.builder.master("local[*]").appName("spark_app").getOrCreate()