David Porter davidporter-id-au

Metrics and alerting

Why monitor?

We need to present a system which is responsible, consistent and reliable to our users. We need therefore, to observe what they are seeing, and be made aware when the services we build are not available.

What to monitor?

So a caveat: My understanding of the macroeconomic climate is hardly different than that of the reading-layman's; I've had little to do with economics since graduation.

The short answer about Australian property prices - to the best of my understanding - is that we're unlikely to see them rise much, but I can't say with any certainty if they're going to fall. If they do fall, it will most likely be on the back of a quite serious macroeconomic malaise. So the problem for any home-buyer in Australia is mostly around an appetite for risk more than anything else: We've no idea how things will play out, but if something dramatic does happen it'll be bad.

The start of the year has been centered around probably three major possible events, all of which are likely to have significant and nasty effects for Australia

2014.... so, I can't really remember much of what happened that year beyond breaking up with my girlfriend. I'm not going to be able to shed all that much light on whats going on there, but to be honest I'd not be all that surprised if the dropbox daemon just died/borked/stopped working or something. I honestly wouldn't try and repair it, it might not work for large amounts of data.

Oddly enough I was kinda doing something similar today with a cronjob, actually the first database backup script I've written here (things move slower). What I'd suggest is rather than caring about the piece-of-shit dropbox software we just dump it into amazon s3 because it's pretty cheap.

All that's needed is to do a mysql dump into a file, compress it with Gzip or something and then you can upload quite simply with the AWS cli tools. The script to run in cron might look something like this:

first, install python pip:

sudo apt-get update && sudo apt-get install python-pip -y

	#!/bin/bash -e

	# A helper script to make finding thigs by AWS tag less annoying
	# DO NOT USE THIS WITH UNTRUSTED INPUT, it's an arbitrary code execution attack
	# waiting to happen.

	if [[ $# -lt 1 ]]; then
	echo "Usage: $0 <tag-key=tag-value> [... tag-key=tag-value]"
	echo ""
	echo "eg: $0 GitBranch=master Application=vision-adapter"

	account required pam_nologin.so
	@include common-account
	session [success=ok ignore=ignore module_unknown=ignore default=bad] pam_selinux.so close
	session required pam_loginuid.so
	session optional pam_keyinit.so force revoke
	@include common-session
	session optional pam_motd.so motd=/run/motd.dynamic
	session optional pam_motd.so noupdate
	session required pam_limits.so
	session required pam_env.so user_readenv=1 envfile=/etc/default/locale

	#!/bin/bash -e

	REGION="ap-southeast-2"

	function waitForStack {

	# to avoid a race condition with AWS, wait a moment for the build to start
	sleep 4

	TIMEOUT_IN_SECONDS=600



	function makePromise(doSomethingAsync){
	return new Promise(function(resolve, reject){
	doSomethingAsync(resolve);
	});
	}

	makePromise(function(cb){
	setTimeout(function(){

	#!/bin/bash -eu
	# Largely based on https://gist.github.com/trevnorris/9616784

	if [[ $# -lt 1 ]]; then
	echo "usage $0 <nodeJS script to graph>"
	exit 1
	fi

	if [[ ! -f ~/sources/FlameGraph/stackcollapse-perf.pl ]]; then
	echo "Needs some setup and scripts. Ensure that you've read/executed https://gist.github.com/trevnorris/9616784"



	(def direction-map ["R3", "R1", "R4", "L4", "R3", "R1", "R1", "L3", "L5", "L5", "L3", "R1", "R4", "L2", "L1", "R3", "L3", "R2", "R1", "R1", "L5", "L2", "L1", "R2", "L4", "R1", "L2", "L4", "R2", "R2", "L2", "L4", "L3", "R1", "R4", "R3", "L1", "R1", "L5", "R4", "L2", "R185", "L2", "R4", "R49", "L3", "L4", "R5", "R1", "R1", "L1", "L1", "R2", "L1", "L4", "R4", "R5", "R4", "L3", "L5", "R1", "R71", "L1", "R1", "R186", "L5", "L2", "R5", "R4", "R1", "L5", "L2", "R3", "R2", "R5", "R5", "R4", "R1", "R4", "R2", "L1", "R4", "L1", "L4", "L5", "L4", "R4", "R5", "R1", "L2", "L4", "L1", "L5", "L3", "L5", "R2", "L5", "R4", "L4", "R3", "R3", "R1", "R4", "L1", "L2", "R2", "L1", "R4", "R2", "R2", "R5", "R2", "R5", "L1", "R1", "L4", "R5", "R4", "R2", "R4", "L5", "R3", "R2", "R5", "R3", "L3", "L5", "L4", "L3", "L2", "L2", "R3", "R2", "L1", "L1", "L5", "R1", "L3", "R3", "R4", "R5", "L3", "L5", "R1", "L3", "L5", "L5", "L2", "R1", "L3", "L1", "L3", "R4", "L1", "R3", "L2", "L2", "R3", "R3", "R4", "R4", "R1", "L4", "R1", "L5"])

	;(de

	#!/bin/bash -e

	# Poor man's installer for nuget
	# Tested only on OSX, other OSs milage may vary

	INSTALL_PATH="/usr/local/lib/nuget"
	BIN_PATH="/usr/local/bin/"

	echo "Installing nuget.exe to $INSTALL_PATH"