Skip to content

Instantly share code, notes, and snippets.

View pganjhu's full-sized avatar

Pawan Kumar Ganjhu pganjhu

4 followers · 0 following
View GitHub Profile
@pganjhu
pganjhu / setup-notes.md
Created April 1, 2022 13:07 — forked from eddies/setup-notes.md
Spark 2.0.0 and Hadoop 2.7 with s3a setup

Standalone Spark 2.0.0 with s3

###Tested with:

  • Spark 2.0.0 pre-built for Hadoop 2.7
  • Mac OS X 10.11
  • Python 3.5.2

Goal

Use s3 within pyspark with minimal hassle.