Skip to content

Instantly share code, notes, and snippets.

View mikasenghaas's full-sized avatar

Mika Senghaas mikasenghaas

View GitHub Profile
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.10"
# dependencies = ["renderers", "transformers>=4.46", "jinja2", "rich"]
# ///
"""
Replay the TITO failure modes from tito_failure_modes.py using
PrimeIntellect's `renderers` package (hand-coded per-model bridges)
instead of the naive `apply_chat_template`-delta algorithm.
#!/usr/bin/env -S uv run --script
# /// script
# requires-python = ">=3.10"
# dependencies = ["transformers", "jinja2", "rich"]
# ///
from transformers import AutoTokenizer
from rich.console import Console
from rich.text import Text
console = Console()
@mikasenghaas
mikasenghaas / oai_pydantic.md
Last active November 1, 2025 00:47
Benchmarks for constructing OAI Pydantic models w/ many logprobs

To reproduce, download the .json files with raw responses from here.

Running in standard mode

uv run oai_pydantic.py
Processing oai_response_1024.json...