Skip to content

Instantly share code, notes, and snippets.

@mkanchwala

mkanchwala/Spark-SingleRDDfrmMultipleFiles

Created April 30, 2015 08:59
Download ZIP

Revisions

  1. Murtaza Kanchwala created this gist Apr 30, 2015.
    6 changes: 6 additions & 0 deletions Spark-SingleRDDfrmMultipleFiles
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,6 @@
    Best way to create Mutliple files into a single RDD
    ==================================

    val fileRDD = sc.textFile(filename).repartition(1)

    Where the filename is the location of your directory only.