Run DeepSeek V4 Flash locally with ds4.c + Pi (10-min onboarding)

ds4-pi-onboarding.md

Local DeepSeek V4 Flash with ds4.c + Pi

A 10-minute setup to run DeepSeek V4 Flash (284B MoE, 1M context) locally as your Pi coding-agent backend. Uses ds4.c as the inference server and Pi as the agent.

What you need

• Mac with Apple Silicon and ≥128 GB unified memory (M3 Max / M4 Max / M-Ultra).
• ~100 GB free disk (81 GB for the q2 GGUF + headroom for the disk KV cache).
• gh / curl / make / Xcode CLT.

1. Build ds4.c

git clone https://github.com/antirez/ds4.c
cd ds4.c
make            # produces ./ds4 (CLI) and ./ds4-server

2. Download the q2 model (~81 GB)

./download_model.sh q2

This pulls the 2-bit asymmetric quant (only routed MoE experts quantized; attention, shared experts, projections stay full-precision) and symlinks ./ds4flash.gguf to it.

3. Smoke test

./ds4 -p "Reply with exactly: pong" --nothink -n 16

First load mmaps 82 GB and asks the kernel for residency — expect ~30–45 s the first time, much faster on subsequent runs while the file is in the page cache.

4. Launch the server

Pick a port that nothing else owns. Port 8000 is a common Docker default — use 8765 to avoid surprises:

./ds4-server --port 8765 \
             --ctx 100000 \
             --kv-disk-dir ~/.cache/ds4-kv \
             --kv-disk-space-mb 8192

• --ctx 100000 is sane for 128 GB. Full 1M context costs ~26 GB extra just for the indexer; the q2 model itself is already 81 GB.
• --kv-disk-dir makes prompt prefixes survive restarts. Pi sends a ~25k-token system preamble each session — caching it on disk turns a 30 s prefill into an instant prefix hit. Use ~/.cache/ds4-kv (persists) over /tmp/ds4-kv (cleared on reboot).

Leave the terminal open. Ctrl+C stops the server.

5. Configure Pi

Add a provider entry to ~/.pi/agent/models.json (preserve any existing providers):

{
  "providers": {
    "ds4": {
      "name": "ds4.c local",
      "baseUrl": "http://127.0.0.1:8765/v1",
      "api": "openai-completions",
      "apiKey": "dsv4-local",
      "compat": {
        "supportsStore": false,
        "supportsDeveloperRole": false,
        "supportsReasoningEffort": true,
        "supportsUsageInStreaming": true,
        "maxTokensField": "max_tokens",
        "supportsStrictMode": false,
        "thinkingFormat": "deepseek",
        "requiresReasoningContentOnAssistantMessages": true
      },
      "models": [
        {
          "id": "deepseek-v4-flash",
          "name": "DeepSeek V4 Flash (ds4.c local)",
          "reasoning": true,
          "thinkingLevelMap": {
            "off": null,
            "minimal": "low",
            "low": "low",
            "medium": "medium",
            "high": "high",
            "xhigh": "xhigh"
          },
          "input": ["text"],
          "contextWindow": 100000,
          "maxTokens": 384000,
          "cost": {"input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0}
        }
      ]
    }
  }
}

contextWindow must be ≤ the --ctx you started the server with.

Optionally make it the default in ~/.pi/agent/settings.json:

{
  "defaultProvider": "ds4",
  "defaultModel": "deepseek-v4-flash"
}

6. Use it

pi --provider ds4 --model deepseek-v4-flash

The first turn pays a one-time prefill on the system preamble; subsequent turns and restarts hit the disk KV cache.

Gotchas

• Don't run ./ds4 and ./ds4-server simultaneously. Each maps the 81 GB GGUF wired; two at once will thrash a 128 GB box.
• Port 8000 is often taken (Docker, dev servers). Verify with lsof -i :PORT -P -n and pick something free.
• Server is single-worker. Concurrent requests serialize on one Metal session.
• Bearer token in ps. download_model.sh passes HF_TOKEN via curl -H, so it's visible in ps output of the curl process while the download runs.
• thinking: {"type":"disabled"} for direct replies; default is thinking mode.

Why ds4.c instead of llama.cpp?

ds4.c is a deliberately narrow, single-model engine: the loader, Metal graph, tokenizer, KV layout, and HTTP server all assume DeepSeek V4 Flash. That buys custom 2-bit asymmetric quantization, on-disk KV checkpoints with SHA1-keyed prefix reuse, OpenAI- and Anthropic-compatible endpoints, and validation against logits captured from the official DeepSeek API. llama.cpp is the right tool for any model; ds4.c is the right tool for this one.