zouzias
/ sparkDataFrameZipWithIndex.scala
Last active
October 28, 2021 14:25
Spark DataFrame zipWithIndex
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import org.apache.spark.sql.Row | |
| import org.apache.spark.sql.types.{StructField,StructType,IntegerType, LongType} | |
| val df = sc.parallelize(Seq((1.0, 2.0), (0.0, -1.0), (3.0, 4.0), (6.0, -2.3))).toDF("x", "y") | |
| // Append "rowid" column of type Long | |
| val newSchema = StructType(df.schema.fields ++ Array(StructField("rowid", LongType, false))) | |
| // Zip on RDD level | |
| val rddWithId = df.rdd.zipWithIndex |
craigminihan
/ gist:b23c06afd9073ec32e0c
Last active
September 21, 2023 12:47
Build GCC 4.9.2 for C/C++ on CentOS 7
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| sudo yum install libmpc-devel mpfr-devel gmp-devel | |
| cd ~/Downloads | |
| curl ftp://ftp.mirrorservice.org/sites/sourceware.org/pub/gcc/releases/gcc-4.9.2/gcc-4.9.2.tar.bz2 -O | |
| tar xvfj gcc-4.9.2.tar.bz2 | |
| cd gcc-4.9.2 | |
| ./configure --disable-multilib --enable-languages=c,c++ | |
| make -j 4 | |
| make install |