veekaybee

Anti-hype LLM reading list

Goals: Add links that are reasonable and good explanations of how stuff works. No hype and no vendor content if possible. Practical first-hand accounts of models in prod eagerly sought.

Foundational Concepts

Pre-Transformer Models

jirihnidek

"""
Example of using sub-parser, sub-commands and sub-sub-commands :-)
"""

import argparse


def main(args):
    """
    Just do something

smurching

import mlflow

# There are two ways to create parent/child runs in MLflow.

# (1) The most common way is to use the fluent
# mlflow.start_run API, passing nested=True:
with mlflow.start_run():
  num_trials = 10
  mlflow.log_param("num_trials", num_trials)
  best_loss = 1e100

hannesdatta

################################################################
# DOWNLOAD ENTIRE FOLDER STRUCTURE FROM DROPBOX TO LOCAL DRIVE #
################################################################

# Instructions:
# (1) install dropbox API using pip
# > pip install dropbox

# (2) Create application to make requests to the Dropbox API
# - Go to: https://dropbox.com/developers/apps

bzerangue

NDJSON is a convenient format for storing or streaming structured data that may be processed one record at a time.

• Each line is a valid JSON value
• Line separator is ‘\n’

1. Convert JSON to NDJSON?

cat test.json | jq -c '.[]' > testNDJSON.json

erdavids

import PIL, random, sys
from PIL import Image, ImageDraw
origDimension = 1500
r = lambda: random.randint(50,215)
rc = lambda: (r(), r(), r())
listSym = []
def create_square(border, draw, randColor, element, size):
  if (element == int(size/2)):
    draw.rectangle(border, randColor)
  elif (len(listSym) == element+1):

HarshTrivedi

import torch
from torch import LongTensor
from torch.nn import Embedding, LSTM
from torch.autograd import Variable
from torch.nn.utils.rnn import pack_padded_sequence, pad_packed_sequence

## We want to run LSTM on a batch of 3 character sequences ['long_str', 'tiny', 'medium']
#
#     Step 1: Construct Vocabulary
#     Step 2: Load indexed data (list of instances, where each instance is list of character indices)

sherjilozair

-- ~celeste~
-- matt thorson + noel berry

-- globals --
-------------

room = { x=0, y=0 }
objects = {}
types = {}
freeze=0

bmcfee

ceshine

class TemporalBlock(tf.layers.Layer):
    def __init__(self, n_outputs, kernel_size, strides, dilation_rate, dropout=0.2, 
                 trainable=True, name=None, dtype=None, 
                 activity_regularizer=None, **kwargs):
        super(TemporalBlock, self).__init__(
            trainable=trainable, dtype=dtype,
            activity_regularizer=activity_regularizer,
            name=name, **kwargs
        )        
        self.dropout = dropout