From 3% to 74%: the lever log of a 28.9M-parameter NPC model (tai/traintai)

new-gist.md

From 3% to 74%: the lever log of a 28.9M-parameter NPC model

What it actually took to make a tiny dense-core model (559K params + a 25M sparse PLE table) run SillyTavern-format game NPCs, measured every step on an adversarial 720-rollout forge. Repos: inference (tai) / training (traintai). The full living version of this log is AGENTS.md in traintai.

The arc in one table

era	what we believed	what measurement showed
"55% pass"	model nearly there	grader blind to template echo: honest pass 3%, echo 81%
"capacity-bound"	559K core too small for depth	the ceiling was 16 hardcoded response templates in the data
"SFT will fix identity"	more tokens	200M tokens never beat ~20% persona swap; 300 GRPO steps -> ~0%
"round 3 regression"	unlucky seed	PG objective summed logprobs; window doubling doubled the scale -> collapse
"flywheel helps"	rejection data is free signal	forge rows ended at the NPC turn -> taught response->eot, no_stop 43%
current	--	74% honest pass, 0% echo, 0% action-beats, sim actions emitting

The levers, ordered by measured impact

1. Grader visibility before training. You cannot improve what the reward cannot see. Every behavior target needs a flaw class / reward term FIRST.
2. Data contamination beats capacity. Any exact string repeated ~1500x in training data is an attractor the model WILL collapse onto. Fix: combinatorial generators, world-DB grounding, decontam filters.
3. Reward coverage gaps hide as plateaus. 4 of 6 question types got full intent reward for free (keys None -> +1). Audit every reward clause against the actual prompt distribution.
4. RL objective scaling. Per-token mean logprob + group-std advantage. Loss spikes >10x baseline = the objective, not the lr.
5. Derived data must preserve the runtime transition. Store forge rows with the turn marker or the model learns response->eot.
6. Identity is a reward problem; form is a data problem. Know which lever class your flaw belongs to before spending compute.
7. The checkpoint owns its prompt convention. Name: beats the training-literal Name: 46% vs 32%. Measure prompt variants.
8. The flywheel compounds. Forge passing rows into the next round's bins: 878 -> 3106 rows drove 42% -> 74% across five rounds.
9. Muon < AdamW at this scale (val 3.44 vs 2.90 matched steps).
10. Decoding is never the depth lever. Temperature grids moved nothing.

The simulation era (current)

An economy simulator is now the decision oracle: stock, demand, markup, haggle floors. Every generated conversation embeds exactly one correct choice, closed by at most one bracket action -- [DEAL: item gold] or [GOTO: place] -- with abstention cases on purpose (out-of-stock means no DEAL, ever). A held-out oracle-labeled scenario set (disjoint seed) measures format rate, action-beats rate, invalid-action rate, and oracle match split by none/GOTO/DEAL. One round =

UV_NO_SYNC=1 uv run python src/round.py --prev runs/<prev>.pt --tag st-rN

Results of the first sim round (r16): dialog-only solved (action-beats 92% -> 0%), actions emitting at 53%, forge held at 74%. The second round (r17) adds the lever that was missing: oracle-labeled action correctness in the GRPO reward.