Lishi He ikbear

Anti-hype LLM reading list

Goals: Add links that are reasonable and good explanations of how stuff works. No hype and no vendor content if possible. Practical first-hand accounts of models in prod eagerly sought.

Foundational Concepts

Pre-Transformer Models

Primer: iOS Design with Figma

Disclaimer: I am definitely not an expert on either design or iOS development.

I recently had to use Figma to do the designs for an iOS app. Here's a quick primer you might find useful.

Topics covered: app screens, icons, iPhone mockups, and App Store previews.

Step 1: App Screens

TL;DR

Install Postgres 10, and then:

sudo pg_dropcluster 10 main --stop
sudo pg_upgradecluster 9.6 main
sudo pg_dropcluster 9.6 main

INTRO

Concurrency is a domain I have wanted to explore for a long time because the locks and the race conditions have always intimidated me. I recall somebody suggesting concurrency patterns in golang because they said "you share the data and not the variables".

Amused by that, I searched for "concurrency in golang" and bumped into this awesome slide by Rob Pike: https://talks.golang.org/2012/waza.slide#1 which does a great job of explaining channels, concurrency patterns and a mini-architecture of load-balancer (also explains the above one-liner).

Let's dig in:

	You are Manus, an AI agent created by the Manus team.

	You excel at the following tasks:
	1. Information gathering, fact-checking, and documentation
	2. Data processing, analysis, and visualization
	3. Writing multi-chapter articles and in-depth research reports
	4. Creating websites, applications, and tools
	5. Using programming to solve various problems beyond development
	6. Various tasks that can be accomplished using computers and the internet

	##################################
	# Data functions
	##################################

	import re

	from datasets import load_dataset

	def clean(text):
	text = text.replace("\n", " ").strip()

	[LLM Training and Fundamentals]
	- GPT and GPT-2: https://cameronrwolfe.substack.com/p/language-models-gpt-and-gpt-2
	- GPT-3 and LLM Scaling: https://cameronrwolfe.substack.com/p/language-model-scaling-laws-and-gpt
	- Modern LLMs: https://cameronrwolfe.substack.com/p/modern-llms-mt-nlg-chinchilla-gopher
	- Specialized LLMs: https://cameronrwolfe.substack.com/p/specialized-llms-chatgpt-lamda-galactica

	[Open Source LLMs]
	- LLaMA: https://cameronrwolfe.substack.com/p/llama-llms-for-everyone
	- Beyond LLaMA (Imitation Models): https://cameronrwolfe.substack.com/p/beyond-llama-the-power-of-open-llms
	- False Promise of Imitation: https://cameronrwolfe.substack.com/p/imitation-models-and-the-open-source

	Summaries and Overviews:
	- GPT and GPT-2: https://cameronrwolfe.substack.com/p/language-models-gpt-and-gpt-2
	- Scaling Laws and GPT-3: https://cameronrwolfe.substack.com/p/language-model-scaling-laws-and-gpt
	- OPT-175B (Open-Source GPT-3): https://cameronrwolfe.substack.com/p/understanding-the-open-pre-trained-transformers-opt-library-193a29c14a15
	- Modern LLMs: https://cameronrwolfe.substack.com/p/modern-llms-mt-nlg-chinchilla-gopher
	- Specialized LLMs: https://cameronrwolfe.substack.com/p/specialized-llms-chatgpt-lamda-galactica
	- Why does ChatGPT work?: https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/
	- Orca: https://cameronrwolfe.substack.com/p/orca-properly-imitating-proprietary
	- LLaMA: https://cameronrwolfe.substack.com/p/llama-llms-for-everyone
	- MPT: https://cameronrwolfe.substack.com/p/democratizing-ai-mosaicmls-impact

	library(httr)
	library(purrr)
	token <- rtweet:::check_token(NULL)
	dsw <- GET(url = "https://api.twitter.com/1.1/lists/members.json?slug=women-in-data-science&owner_screen_name=becomingdatasci&cursor=-1&count=5000&skip_status=1", token)
	map_chr(content(dsw)$users, "screen_name") %>% sort %>% cat(sep = "\n")
	#> ___dhara___
	#> __Helma
	#> _ajbc
	#> _beenkim
	#> _ehinosa

	package main

	import (
	"context"
	"flag"
	"fmt"
	"log"
	"net/http"
	"os"
	"os/signal"