Running uncensored / reasoning Qwen3.6 models on a single RTX 3090 Ti (24GB VRAM) via llama.cpp's OpenAI-compatible server, wired into OpenCode.
Current primary: Qwopus3.6-27B (CoT fine-tune, working MTP) at 128K context — more accurate on code, ~68-108 tok/s. Fast fallback: Qwen3.6-35B-A3B Wasserstein (uncensored MoE) at 224K — ~174 tok/s.
Note: Qwopus is the one Qwen3.6 GGUF where MTP speculative decoding works in llama.cpp (it self-drafts from embedded heads). It must run with thinking OFF and temperature 0.9 — the opposite of base Qwen3.6, which over-thinks and loops if thinking is left on.
- GPU: NVIDIA RTX 3090 Ti (24,564 MiB VRAM)
- OS: Linux (Pop!_OS), headless
- llama.cpp build: 185 (c2ba3e4, 2026-06-14)
- File: Qwopus3.6-27B-v2-MTP-IQ4_XS.gguf (~15.4 GB)
- Architecture: Qwen3.6-27B dense (~27B active params/token)
- Fine-tune: Chain-of-Thought traces for better reasoning/coding accuracy
- MTP: the one Qwen3.6 GGUF where llama.cpp speculative decoding works (self-drafts from embedded heads, ~65-95% acceptance)
- Operating notes: run thinking OFF + temperature 0.9 (opposite of base Qwen3.6 — leaving thinking on causes it to loop). 128K context ceiling (MTP draft-context VRAM overhead). ~68-108 tok/s.
- Source: https://huggingface.co/Jackrong/Qwopus3.6-27B-v2-MTP-GGUF
- File: Qwen3.6-35B-A3B-Uncensored.IQ4_XS.gguf (~18.7 GB)
- Architecture: MoE, 35B total / 3B active per token
- Abliteration: Wasserstein method (uncensored; better capability preservation than aggressive abliteration)
- Speed/context: ~174 tok/s, 224K context. Use when speed or uncensored output matters more than coding accuracy.
- Source: https://huggingface.co/LuffyTheFox/Qwen3.6-35B-A3B-Uncensored-Wasserstein-GGUF
| Component | VRAM |
|---|---|
| Model weights IQ4_XS | ~12.2 GB |
| KV cache 224K q8_0 | ~9.6 GB |
| Compute / overhead | ~0.8 GB |
| Total | ~21.1 GB |
| Free headroom | ~2.9 GB |
-c 229376 (224K context)
Empirically tested the ceiling on this card:
- 224K → 21.1GB used, 2.9GB free ✅
- 256K → 21.6GB used, 2.5GB free (too tight under load)
The model's trained max is 262144 (256K) — going beyond that degrades quality regardless of VRAM. 224K is the practical safe maximum on 24GB.
--cache-type-k q8_0 --cache-type-v q8_0
8-bit KV cache instead of fp16. At 224K context, fp16 KV cache would cost ~19GB
alone — impossible alongside 12GB of weights. q8_0 cuts it to ~9.6GB with
negligible quality loss.
--flash-attn on
Reduces peak VRAM spikes during long-context inference and speeds up generation
on sequences > 8K tokens. Required at this context size.
--parallel 1
One concurrent slot. With ~2.9GB headroom, adding parallel slots would OOM.
-ngl 99
All 99 transformer layers on GPU. Without full offload, inference drops from
~170 tok/s to ~2 tok/s.
--jinja
Required for Qwen3 chat template. Without it, <think> tokens and tool calls
are not formatted correctly.
KillSignal=SIGINT
llama-server handles SIGINT gracefully (flushes KV cache). SIGTERM causes
unclean shutdown.
- Generation: ~171 tok/s
- Prompt eval: ~347 tok/s
- No speculative decoding (see note below)
Qwen3.6 "Native-MTP-Preserved" GGUFs embed MTP heads for vLLM/SGLang only.
llama.cpp's --spec-type draft-mtp requires a separate mtp-*.gguf sibling
file (like Google's Gemma-4 ships), which no Qwen3.6 repo currently publishes.
The EAGLE3 speculative path exists in llama.cpp but is currently unstable for
Qwen3.6 (crashes past ~700 tokens, issue #24541).
If you need speculative decoding with Qwen3.6, use vLLM:
--speculative-config '{"method":"qwen3_next_mtp","num_speculative_tokens":2}'
# Install unit
sudo cp llama-server.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now llama-server
# Status / logs
systemctl status llama-server
journalctl -u llama-server -f
# Verify live model and context
curl -s http://localhost:8080/v1/models | \
python3 -c "import sys,json; m=json.load(sys.stdin)['data'][0]; print(m['id'], 'ctx:', m['meta']['n_ctx'])"
# Clean stop (matches binary, not wrapper shell)
pkill -INT -x llama-server