Skip to content

Instantly share code, notes, and snippets.

@filipmihal
filipmihal / README.md
Created September 8, 2026 12:13
vLLM hybrid Mamba + MTP corruption reproducer (reorder threshold)

Reproducer: hybrid Mamba + MTP corruption from a lowered batch-reorder threshold (vLLM 0.27.1)

Files

  • probe_sitecustomize.py — put its directory on PYTHONPATH of vllm serve and set RACE_PROBE=log,geom RACE_PROBE_LOG=/path/probe.jsonl. Logs the runner's reorder_batch_threshold, every builder's threshold, the rows of every Mamba2 prefill step ([request, num_computed_before, tokens_this_step]) and every batch swap. No behaviour change. RACE_PROBE=log,geom,reorderfix additionally forces the runner threshold to 1 + k (the monkeypatch used as the positive control).
  • burst.py — load generator. `--mode pairs --pair-long-reps 1200 --pair-delay 1.5 --pair-delay-jitter 1.0