Created
May 21, 2019 11:42
-
-
Save afranzi/499c2a4cb389e9247f6d219549994e30 to your computer and use it in GitHub Desktop.
PySpark with org.apache.hadoop:hadoop-aws:2.8.5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| export SPARK_OPTS='--packages org.apache.hadoop:hadoop-aws:2.8.5 --conf spark.hadoop.fs.s3a.aws.credentials.provider=org.apache.hadoop.fs.s3a.TemporaryAWSCredentialsProvider' | |
| pyspark |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment