Alex bzz

Built-in

diskutil erasevolume HFS+ 'RAM Disk' `hdiutil attach -nobrowse -nomount ram://XXXXX`

where XXXXX is the size of the RAM disk in terms of memory blocks.

Notes:

tmux shortcuts & cheatsheet

start new:

tmux

start new with session name:

tmux new -s myname

# Delete all containers
docker rm $(docker ps -aq)
# Delete all images
docker rmi $(docker images -q)
# Delete all untagged images
docker rmi $(docker images -q --filter "dangling=true")

References:

Easiest HDFS cluster in the world with kubernetes.

Inspiration from kimoonkim/kubernetes-HDFS

kubectl create -f namenode.yaml
kubectl create -f datanode.yaml

Setup a port-forward to so you can see it is alive:

Install

Either copy the aliases from the .gitconfig or run the commands in add-pr-alias.sh

Usage

Easily checkout local copies of pull requests from remotes:

git pr 4 - creates local branch pr/4 from the github upstream(if it exists) or origin remote and checks it out
git pr 4 someremote - creates local branch pr/4 from someremote remote and checks it out

Generating Flame Graphs for Apache Spark

Flame graphs are a nifty debugging tool to determine where CPU time is being spent. Using the Java Flight recorder, you can do this for Java processes without adding significant runtime overhead.

When are flame graphs useful?

Shivaram Venkataraman and I have found these flame recordings to be useful for diagnosing coarse-grained performance problems. We started using them at the suggestion of Josh Rosen, who quickly made one for the Spark scheduler when we were talking to him about why the scheduler caps out at a throughput of a few thousand tasks per second. Josh generated a graph similar to the one below, which illustrates that a significant amount of time is spent in serialization (if you click in the top right hand corner and search for "serialize", you can see that 78.6% of the sampled CPU time was spent in serialization). We used this insight to spee

Interdisciplinary Open Source Community Conferences

Criteria

Must be an event that someone involved in open source would be interested in attending
Must be a community oriented event (no corporate owned for-profit events here please)
Can't be about a specific language/framework.

Leave suggestions in the comments below

	#!/bin/bash

	# From http://tech.serbinn.net/2010/shell-script-to-create-ramdisk-on-mac-os-x/
	#

	ARGS=2
	E_BADARGS=99

	if [ $# -ne $ARGS ] # correct number of arguments to the script;
	then

	-- Active issues
	-- Count of total active issues in the specified time frame
	-- Source: githubarchive public data set via Google BigQuery http://githubarchive.org/

	SELECT
	COUNT(DISTINCT JSON_EXTRACT_SCALAR(events.payload, '$.issue.id')) AS events_issue_count
	FROM (SELECT * FROM TABLE_DATE_RANGE([githubarchive:day.],TIMESTAMP('2015-09-01'),TIMESTAMP('2016-08-31')))
	AS events

	-- 10,723,492 active issues

	#!/boot/bzImage

	# Linux kernel userspace initialization code, translated to bash
	# (Minus floppy disk handling, because seriously, it's 2017.)
	# Not 100% accurate, but gives you a good idea of how kernel init works
	# GPLv2, Copyright 2017 Hector Martin <[email protected]>
	# Based on Linux 4.10-rc2.

	# Note: pretend chroot is a builtin and affects the current process
	# Note: kernel actually uses major/minor device numbers instead of device name