Skip to content

Instantly share code, notes, and snippets.

@runlevel5
Last active June 30, 2026 02:52
Show Gist options
  • Select an option

  • Save runlevel5/bbe1afdef946b716f83b978bd6cc57f5 to your computer and use it in GitHub Desktop.

Select an option

Save runlevel5/bbe1afdef946b716f83b978bd6cc57f5 to your computer and use it in GitHub Desktop.
quake3e JIT: value left live on the opStack is corrupted across a taken conditional branch (all backends)

JIT: value left live on the opStack is corrupted across a taken conditional branch (all backends)

Summary

When a value is left live on the QVM opStack across a conditional branch and the branch is taken, the compiled VM can return a wrong result. The bytecode interpreter produces the correct value. Reproduces on current master (fe57ee9) on all three JIT backends — x86_64 (vm_x86.c), aarch64 (vm_aarch64.c) and ppc64le (vm_powerpc.c) — deterministically, with a 13-instruction function: no data input, no syscalls, no undefined behaviour.

Environment

All verified at upstream commit fe57ee9, -O2, against the reference code/qcommon/vm_interpreted.c (correct on every platform):

  • x86_64 — Linux x86_64, gcc 15.2.0 — backend code/qcommon/vm_x86.c
  • aarch64 — Linux aarch64, gcc 15.2.1 — backend code/qcommon/vm_aarch64.c
  • ppc64le — Linux ppc64le, gcc 16.1.1 — backend code/qcommon/vm_powerpc.c

Reproduction

Drop repro.c (below) at the repo root and build it against the VM units, picking the backend translation unit. From the repo root:

cc -O2 -ffunction-sections -fdata-sections -Wl,--gc-sections -Icode/qcommon \
   repro.c code/qcommon/vm.c code/qcommon/vm_interpreted.c \
   code/qcommon/vm_x86.c -lm -o repro && ./repro      # or vm_aarch64.c / vm_powerpc.c

Observed output (deterministic across runs):

test                                   x86_64       aarch64      ppc64le
A  one const live,        taken        ok           ok           ok
B  const + reg live,      taken        0x7f  BAD    0x10c BAD    0x7f  BAD   (expected 0x8d)
D  two reg values live,   taken        0x0   BAD    ok           0x0   BAD   (expected 0xcccccccc)
E  const + reg live,      NOT taken    ok           ok           ok

The failing case (test B — fails on every backend)

QVM function (one proc), in QVM opcodes:

ENTER 0x100
CONST 0x7f          ; slot1 = 127            (left live on the opStack)
CONST 100
CONST 7
DIVI                ; slot2 = 100/7 = 14     (left live on the opStack)
CONST 0
CONST -2
GTI   -> ADD        ; if (0 > -2) goto ADD   — condition TRUE, branch is taken
ADD                 ; slot1 + slot2
LEAVE 0x100
  • Expected / interpreter: 127 + 14 = 141 (0x8d)
  • x86_64 / ppc64le JIT: 127 (0x7f) — slot2 lost
  • aarch64 JIT: 268 (0x10c)

The interpreter result is provably correct and the program has no undefined behaviour (100/7 is a well-defined non-overflowing divide; no shift, no out-of-range conversion, no out-of-bounds access). So this is a JIT codegen defect, with the interpreter as the reference.

What isolates the trigger

Test opStack values live across the branch taken? x86_64 aarch64 ppc64le
A one CONST yes ok ok ok
B one CONST + one register value (DIVI) yes BAD BAD BAD
D two register values (NEGI,BCOM) yes BAD ok BAD
E same as B no ok ok ok

The corruption requires a value left live on the opStack across a taken conditional branch; a not-taken branch is always fine. x86_64 and ppc64le are broader (they also lose two register-resident values, test D); aarch64 fails test B only. It is not specific to the CONST;CMP ConstOptimize fast path — using a computed compare operand (so the main comparison handler is used) reproduces test B identically.

Impact

A genuine JIT miscompilation on every supported backend: a conditional branch can yield a wrong arithmetic result depending on control flow, with no undefined behaviour involved. It is latent with respect to today's content (see below) — but it is a real codegen defect, reachable by any bytecode that leaves a value on the opStack across a branch.

Why this went unnoticed

All shipping QVMs are produced by lcc/q3asm, whose code generator keeps the opStack empty between statements: values that live across control flow are spilled to stack-frame locals in memory (OP_LOCAL + OP_STORE/OP_LOAD), never held on the opStack. So a conditional branch is always reached with an otherwise-empty opStack — just the two operands being compared. The JIT's opStack register-caching optimizer silently depends on this: it never reconciles register state across a branch, because in real bytecode nothing is ever live there to reconcile.

Verified empirically — scanning per-instruction opStack depth (via the engine's own VM_LoadInstructions) across 12 shipping QVMs: Quake3e + ioquake3 baseq3/missionpack and CPMA (cgame/qagame/ui), ~1.6M instructions, ~51,000 conditional compares and ~30,000 jumps. Every compare and jump is reached with exactly its operands on the opStack and nothing below it (max opStack at any compare = 8 bytes = two operands; zero "live-across" sites in any file).

The QVM format permits a value to be live on the opStack across a branch, but no current compiler emits that — so today's games do not trigger it. It is a latent codegen correctness bug (reachable by hand-written bytecode, a different/future compiler, or a change to lcc) surfaced by a differential interpreter-vs-JIT fuzzer that synthesizes such inputs.

repro.c

repro.c (self-contained; same file builds against vm_x86.c / vm_aarch64.c / vm_powerpc.c)
/*
 * Self-contained reproducer: the QVM JIT corrupts a value left live on the
 * opStack across a *taken* conditional branch. Reproduces on x86_64 (vm_x86.c),
 * aarch64 (vm_aarch64.c) and ppc64le (vm_powerpc.c).
 *
 * Each test builds a tiny ENTER/.../LEAVE QVM function (no data input, no
 * syscalls, fully deterministic) and runs it through BOTH the bytecode
 * interpreter (the reference) and the JIT, printing both results.
 *
 * Build (from the repo root) — pick the backend translation unit:
 *   cc -O2 -ffunction-sections -fdata-sections -Wl,--gc-sections -Icode/qcommon \
 *      repro.c code/qcommon/vm.c code/qcommon/vm_interpreted.c \
 *      code/qcommon/vm_x86.c   -lm -o repro   # or vm_aarch64.c / vm_powerpc.c
 *   ./repro
 */
#include "vm_local.h"
#include <stdio.h>
#include <stdlib.h>
#include <stdarg.h>
#include <string.h>
#include <stdint.h>

/* --- minimal engine stubs needed to link vm.c / vm_interpreted.c / vm_aarch64.c --- */
extern cvar_t *vm_rtChecks;
static cvar_t  rtc;
void QDECL Com_Printf ( const char *fmt, ... ) { (void)fmt; }
void QDECL Com_DPrintf( const char *fmt, ... ) { (void)fmt; }
void NORETURN QDECL Com_Error( errorParm_t lvl, const char *fmt, ... ) {
	va_list ap; (void)lvl; va_start(ap,fmt); vfprintf(stderr,fmt,ap); va_end(ap); fputc('
',stderr); exit(3);
}
void *Hunk_Alloc( size_t n, ha_pref p ) { (void)p; return calloc(1,n); }
int   Hunk_MemoryRemaining( void ) { return 0x7fffffff; }
void *Z_Malloc( size_t n ) { return calloc(1,n); }
void  Z_Free( void *p ) { free(p); }
int   CPU_Flags = 0x3F; /* x86 backend only: FCOM|MMX|SSE|SSE2|SSE3|SSE41 (modern x86_64); unused on arm/ppc */
static intptr_t syscall( intptr_t *a ) { (void)a; return 0; }

/* --- QVM assembler --- */
static byte cb[256]; static int cl, ic;
static void o0(byte op){ cb[cl++]=op; ic++; }
static void o4(byte op,int32_t v){ cb[cl++]=op; memcpy(cb+cl,&v,4); cl+=4; ic++; }

/* A: one CONST value live across a taken branch (skipped store block)        -> ok */
static void A(void){ o4(OP_ENTER,0x100); o4(OP_CONST,0xAAAAAAAA);
	o4(OP_CONST,0); o4(OP_CONST,-2); o4(OP_GTI,8); o4(OP_CONST,0x1000); o4(OP_CONST,0x55); o0(OP_STORE4);
	o4(OP_LEAVE,0x100); o0(OP_PUSH); o4(OP_LEAVE,0x100); }
/* B: CONST 127 + register value (100/7=14) both live, taken branch, then ADD -> BUG */
static void B(void){ o4(OP_ENTER,0x100); o4(OP_CONST,0x7f);
	o4(OP_CONST,100); o4(OP_CONST,7); o0(OP_DIVI);
	o4(OP_CONST,0); o4(OP_CONST,-2); o4(OP_GTI,8); o0(OP_ADD);
	o4(OP_LEAVE,0x100); o0(OP_PUSH); o4(OP_LEAVE,0x100); }
/* D: two register values live, taken branch (skipped store)                  -> ok */
static void D(void){ o4(OP_ENTER,0x100); o4(OP_CONST,0x11111111); o0(OP_NEGI);
	o4(OP_CONST,0x22222222); o0(OP_BCOM);
	o4(OP_CONST,0); o4(OP_CONST,-2); o4(OP_GTI,11); o4(OP_CONST,0x1000); o4(OP_CONST,0x55); o0(OP_STORE4);
	o0(OP_ADD); o4(OP_LEAVE,0x100); o0(OP_PUSH); o4(OP_LEAVE,0x100); }
/* E: same live state as B but branch NOT taken (condition false)             -> ok */
static void E(void){ o4(OP_ENTER,0x100); o4(OP_CONST,0x7f);
	o4(OP_CONST,100); o4(OP_CONST,7); o0(OP_DIVI);
	o4(OP_CONST,0); o4(OP_CONST,2); o4(OP_GTI,8); o0(OP_ADD);
	o4(OP_LEAVE,0x100); o0(OP_PUSH); o4(OP_LEAVE,0x100); }

static void setup( vm_t *vm, vmHeader_t *h ){
	unsigned int dl,da; memset(vm,0,sizeof(*vm)); vm->name="repro"; vm->systemCall=syscall;
	vm->exactDataLength=h->bssLength; dl=vm->exactDataLength; if(dl<PROGRAM_STACK_SIZE)dl=PROGRAM_STACK_SIZE;
	vm->programStackExtra=PROGRAM_STACK_EXTRA;
	if(log2pad(dl,1)-dl>=PROGRAM_STACK_EXTRA)vm->programStackExtra=log2pad(dl,1)-dl; else dl+=vm->programStackExtra;
	vm->dataLength=dl; dl=log2pad(dl,1); da=dl+VM_DATA_GUARD_SIZE;
	vm->dataBase=calloc(da,1); vm->dataMask=dl-1; vm->dataAlloc=da;
	vm->instructionCount=h->instructionCount; vm->codeLength=h->codeLength;
	vm->programStack=vm->dataMask+1; vm->stackBottom=vm->programStack-PROGRAM_STACK_SIZE-vm->programStackExtra;
}

static int run( const char *name, void(*bld)(void), int32_t expect ){
	static byte img[sizeof(vmHeader_t)+256]; vmHeader_t *h=(vmHeader_t*)img; int off=sizeof(vmHeader_t);
	vm_t I,C; int32_t rI,rC;
	cl=0; ic=0; bld();
	memset(h,0,sizeof(*h)); h->vmMagic=VM_MAGIC; h->instructionCount=ic; h->codeOffset=off;
	h->codeLength=cl; h->dataOffset=off+cl; h->bssLength=0x40000; memcpy(img+off,cb,cl);
	setup(&I,h); VM_PrepareInterpreter2(&I,h); rI=VM_CallInterpreted2(&I,0,NULL);
	setup(&C,h); C.compiled = VM_Compile(&C,h); rC=VM_CallCompiled(&C,0,NULL);
	printf("%-2s  expect=0x%08x  interpreter=0x%08x  JIT=0x%08x  %s
", name,
		(uint32_t)expect,(uint32_t)rI,(uint32_t)rC, (rI==rC)?"ok":"*** JIT DIVERGES ***");
	return rI!=rC;
}

int main(void){
	int bad=0;
	vm_rtChecks=&rtc; rtc.integer=15;            /* all runtime checks on */
	bad += run("A", A, 0xAAAAAAAA);              /* 1 const live, taken          */
	bad += run("B", B, 0x8d /*141*/);            /* const + reg live, taken      */
	bad += run("D", D, 0xCCCCCCCC);              /* 2 regs live, taken           */
	bad += run("E", E, 0x8d /*141*/);            /* const + reg live, NOT taken  */
	return bad ? 1 : 0;
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment