yanakad

Profiling performance

Using perf:

$ perf record -g binary
$ perf script | stackcollapse-perf.pl | rust-unmangle | flamegraph.pl > flame.svg

NOTE: See @GabrielMajeri's comments below about the -g option.

yanakad

// Colors reference
// You can use the following as so:
// console.log(colorCode, data);
// console.log(`${colorCode}some colorful text string${resetCode} rest of string in normal color`);
// 
// ... and so on.

export const reset = "\x1b[0m"
export const bright = "\x1b[1m"
export const dim = "\x1b[2m"

yanakad

Byobu Commands
==============

        byobu                              Screen manager


Level 0 Commands (Quick Start)
------------------------------

        <F2>                               Create a new window

yanakad

import kafka.server.KafkaConfig;
import kafka.server.KafkaServer;

import java.io.File;
import java.io.FileNotFoundException;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Properties;

yanakad

Boston Spark October 2015 Meetup

Boston Spark Community Updates

• 1000+ members; Meetup started in May 2014.
• Speakers: We want you!
  • Feature talks or Lighting talks
• Sponsors: You want us!
  • Host an event, pay for food and drink, or cover other event costs.
• Take a few minutes at the start of the event to pitch your company, job, or product.

yanakad

1. General Background and Overview

• Probabilistic Data Structures for Web Analytics and Data Mining : A great overview of the space of probabilistic data structures and how they are used in approximation algorithm implementation.
• Models and Issues in Data Stream Systems
• Philippe Flajolet’s contribution to streaming algorithms : A presentation by Jérémie Lumbroso that visits some of the hostorical perspectives and how it all began with Flajolet
• Approximate Frequency Counts over Data Streams by Gurmeet Singh Manku & Rajeev Motwani : One of the early papers on the subject.
• [Methods for Finding Frequent Items in Data Streams](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.187.9800&rep

yanakad

registered table thread_459                          <--Register table gets called
 dropped table thread_259
 dropped table thread_759
 registered table thread_860
 dropped table thread_860
 registered table thread_560
 registered table thread_060
 registered table thread_960
 registered table thread_160
 dropped table thread_560

yanakad

yanakad

wget http://www.eng.lsu.edu/mirrors/apache/spark/spark-1.2.1/spark-1.2.1-bin-hadoop2.3.tgz
tar -xf spark-1.2.1-bin-hadoop2.3.tgz
cd spark-1.2.1-bin-hadoop2.3/bin/
wget https://oss.sonatype.org/content/repositories/snapshots/org/elasticsearch/elasticsearch-hadoop/2.1.0.BUILD-SNAPSHOT/elasticsearch-hadoop-2.1.0.BUILD-20150324.023417-341.jar
 ./spark-shell --jars elasticsearch-hadoop-2.1.0.BUILD-20150324.023417-341.jar
 
import org.apache.spark.sql.SQLContext

case class KeyValue(key: Int, value: String)
val sqlContext = new org.apache.spark.sql.SQLContext(sc)

yanakad