mahedi-kaysar

System Design Cheatsheet

Picking the right architecture = Picking the right battles + Managing trade-offs

Basic Steps

1. Clarify and agree on the scope of the system

• User cases (description of sequences of events that, taken together, lead to a system doing something useful)
  • Who is going to use it?
  • How are they going to use it?

mahedi-kaysar

val ssc = new StreamingContext("local", "datastream", Seconds(15))

// create InputDStream

ssc.registerInputStream(stream)

// interact with stream

ssc.start()

mahedi-kaysar

import org.apache.spark.sql.{ SQLContext, Row }
val sqlContext = new SQLContext(sc)
import sqlContext._


/*
Next you'll need to get your data registered as a table.  Note that in order to infer the schema (column names and types) it must be a case class.  A normal class won't work, unless you manually implement much of what a case class would do, e.g. extend Product and Serializable.

Any data source can be used, as long as you can transform it to an rdd of case classes.  For now, we'll just use some dummy data:
*/

mahedi-kaysar

//This sample is how to use websocket of Tomcat.
package wsapp;

import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.CharBuffer;
import java.util.ArrayList;
import org.apache.catalina.websocket.MessageInbound;
import org.apache.catalina.websocket.StreamInbound;
import org.apache.catalina.websocket.WebSocketServlet;