Joel Carter JoelBCarter

Extracting financial disclosure reports and police blotter narratives using OpenAI's Structured Output

tl;dr this demo shows how to call OpenAI's gpt-4o-mini model, provide it with URL of a screenshot of a document, and extract data that follows a schema you define. The results are pretty solid even with little effort in defining the data — and no effort doing data prep. OpenAI's API could be a cost-efficient tool for large scale data gathering projects involving public documents.

OpenAI announced Structured Outputs for its API, a feature that allows users to specify the fields and schema of extracted data, and guarantees that the JSON output will follow that specification.

For example, given a Congressional financial disclosure report, with assets defined in a table like this:

I've designed a lot of RPC protocols in my career. One pattern that's worked well basically goes as follows:

// Client calls: print('Hello World\n')
-> [1, "print", "Hello World!\n"]
// Server sends return value (or lack of return vvalue)
<- [-1]

// Client calls: add(1, 2)
-> [2, "add", 1, 2]

Kubernetes + AWS ECR = ❤️

Makefile and YAML templates for automating the use of AWS Elastic Container Registry with Kubernetes.

Based off of this awesome Redsaid blog post.

Requirements

Amazon ECR, along with your AWS account ID and the region your ECR is in
AWS CLI

System Design Cheatsheet

Picking the right architecture = Picking the right battles + Managing trade-offs

Basic Steps

Clarify and agree on the scope of the system

User cases (description of sequences of events that, taken together, lead to a system doing something useful)
- Who is going to use it?
- How are they going to use it?

The Ultimate Unit Testing Cheat-sheet

Sinon Chai
Chai
Sinon

Setup modern.ie vagrant boxes

Since modern.ie released vagrant boxes, it' no longer necessary to manually import the ova file to virtualbox, as mentioned here.

However, the guys at modern.ie didn't configured the box to work with WinRM. This how-to addresses that, presenting steps to proper repackage these boxes, adding WinRM support. Additionally configures chocolatey package manager and puppet provisioner.

Pre-requisites

ZeroMQ vs Redis

This document is research for the selection of a communication platform for robot-net.

Goal

The purpose of this component is to enable rapid, reliable, and elegant communication between the various nodes of the network, including controllers, sensors, and actuators (robot drivers). It will act as the core of robot-net to create a standardized infrastructure for robot control.

Requirements:

	// https://github.com/deebloo/rxjs-worker
	var observable = Observable.of([0, 1, 2, 3, 4]);

	observable
	.map(function (data) {
	return data.concat([5, 6, 7, 8, 9]);
	})
	.workerMap(function (data) {
	return data.concat([10,11,12,13,14]);;
	})

	// Run this in the F12 javascript console in chrome
	// if a redirect happens, the page will pause
	// this helps because chrome's network tab's
	// "preserve log" seems to technically preserve the log
	// but you can't actually LOOK at it...
	// also the "replay xhr" feature does not work after reload
	// even if you "preserve log".

	window.addEventListener("beforeunload", function() { debugger; }, false)

	# PostgreSQL can be on a remote server but you'll need root privileges in Linux and superuser in PostgreSQL.

	# First install build tools

	sudo su
	aptitude install build-essential
	aptitude install postgresql-server-dev-9.4

	# Clone and build the PL/pgSQL server-side debugger