Graydon Hoare graydon

Purpose

Bootstrap knowledge of LLMs ASAP. With a bias/focus to GPT.

Avoid being a link dump. Try to provide only valuable well tuned information.

Prelude

Neural network links before starting with transformers.

Hello Friends,

This elf begging to climb onto the web for Christmas began as a personal email, a review of Paul Graham's little Lisp Bel. He sprouted arms, legs, and in gingerstyle ran away. Arms for symbols, legs for conses: these primitives are the mark a Lisp — even more so than the parenthesis. What do we get when we remove these foundation stones: naming and pairing?

No pairs. No cons. No structure. Unordered. Chaos. Eve, a beautifully incomplete aspect oriented triple store. No need for legs when you can effortlessly transport to your destination. Lazy. Pure. Here and now, a retrospective.

No symbols. No names. No variables. Combinators. Forth. No need for arms when you can effortlessly push and pop your stack. No words. A world without words. Virtual worlds. Virtual reality. Space. Time. Motion. Action. Kinetic Programming, a proposal.

I apologize in advance. Checking my pocketwatch, I see I haven't t

	This is a small note about a delightful function. Not cryptography advice or serious commentary. Just amusement.

	A couple years back I had occasion to read in slightly more detail than I had before about the state of the art in cryptographically secure PRNGs (CSPRNGs). These are PRNGs we trust to have additional properties beyond the speed and randomness requirements of normal ones -- inability for an attacker to reveal internal state, mainly, so you can use them to generate secrets.

	If you look, you'll find a lot of people recommending something based on one of Dan Bernstein's algorithms: <a href="https://en.wikipedia.org/wiki/Salsa20">Salsa20 or ChaCha (or even more obscurely "Snuffle")</a>. All the algorithms we're discussing here are very similar in design, and vary only in minor details of interest only to cryptographers.

	If you follow that link though, you'll notice it's a description of a (symmetric) stream cipher. Not a CSPRNG at all!

	But that's ok! Because it turns out that people have long known

	Elsewhere I've been asked about the task of replaying the bootstrap process for rust. I figured it would be fairly straightforward, if slow. But as we got into it, there were <em>just</em> enough tricky / non-obvious bits in the process that it's worth making some notes here for posterity.

	<h3>context</h3>
	Rust started its life as a compiler written in ocaml, called <b>rustboot</b>. This compiler did <em>not</em> use LLVM, it just emitted 32-bit i386 machine code in 3 object file formats (Linux PE, macOS Mach-O, and Windows PE).

	We then wrote a <em>second</em> compiler <em>in Rust</em> called <b>rustc</b> that <em>did</em> use LLVM as its backend (and which, yes, is the genesis of today's rustc) and ran rustboot on rustc to produce a so-called "stage0 rustc". Then stage0 rustc was fed the sources of rustc again, producing a stage1 rustc. Successfully executing this stage0 -> stage1 step (rather than just crashing mid-compilation) is what we're going to call "bootstrapping". There's also a third step: runnin

	# this file isn't a full/complete grammar or even
	# a correct one, but it contains the phrase chunks
	# that I've found useful to practice / focus on
	# while learning talon, I keep it open on screen

	# alphabet
	air bat cat drum
	each fine gust harp
	sit jury krunch look
	made near odd pit

	(* The magic of ocaml polymorphic variants:
	*
	* - subsystem A produces an error `NSaServ
	* - subsystem B checks for error `SaupDOF
	* - both exist in a world with some extra variant `Common
	*
	* the two odd symbols _hash to the same representation_ in ocaml's
	* implementation of polymorphic variant tagging.
	*)

	// This is just an elaboration of an off-hand toot I made earlier today
	// concerning a coding pattern I find myself doing whenever possible: passing
	// and returning owned values instead of borrowing references.
	//
	// I find it works well for long-lived values especially since the resulting
	// composite objects have no lifetime qualifiers, so I don't have to plumb
	// lifetimes through to all the code that uses them.

	// Assumption: assume we have a few objects like this: a network connection, a
	// database, some commands, etc. and we want to make a session type that uses


	common stuff
	-----------


	Basic consonants(32):
	कखगघङचछजझटठडढणतथदधनपफबभमयरलवशषसह

	Weirdo that only is used in ligatures, but necessary(1)
	ञ

	Supporting examples 90 words (9 examples @ 10 words each)
	Elevator pitch 50 words
	Positioning statement 35 words
	Headline benefits 24 words (3 benefits @ 8 words each)
	Mission statement 20 words
	Brand pillars 15 words (3 pillars @ 5 words each)
	Target audience 15 words
	Brand promise 10 words

	{-# LANGUAGE FlexibleInstances #-}
	module MiniToneInference where

	import Prelude hiding ((&&))
	import Control.Applicative
	import Control.Monad
	import Data.Map.Strict (Map, (!))
	import Data.Maybe (fromJust)
	import Data.Monoid ((<>))
	import Data.Set (isSubsetOf)