dharadhruve

dharadhruve / flatten.java

Last active July 18, 2019 08:15 — forked from ebuildy/flatten.java

Flatten Spark data frame fields structure, via SQL in Java. This fork also supports ArrayType fields.

	class Toto
	{
	public void Main()
	{
	final DataFrame source = GetDataFrame();
	final String querySelectSQL = flattenSchema(source.schema(), null);

	source.registerTempTable("source");

	final DataFrame flattenData = sqlContext.sql("SELECT " + querySelectSQL + " FROM source")