local llm openspec workflow

README.md

OpenSpec Local Two-Model Workflow

A spec-driven development setup for running OpenSpec against two local models via llama.cpp, built to fight the most common local-model failure: unchecked assumptions against the current implementation.

The whole design converts silent assumptions into things that must survive three confrontations with reality — being written down, being fact-checked by a second model, and being executed by tests — before code is trusted.

The two-model split

Model	Role	Why	Server
Gemma 4 26B-A4B (QAT, MoE ~4B active)	explore, propose	Fits a 16GB card, fast and interactive for the read-and-draft loop	localhost:8080 id gemma4
Qwen3.6 35B-A3B (MoE ~3.8B active)	verify, apply	Stronger reasoning / agentic coding; used to fact-check the other model's plan and to implement	localhost:8081 id qwen36

Running verification on the opposite model from the one that wrote the proposal is the point: the checker has no investment in the plan, so it won't wave through a convenient assumption.

What's in this package

.
├── Makefile                     # launches both models with correct aliases/ports
├── AGENTS.md                    # apply-phase discipline (read by your coding agent)
├── README.md                    # this file
└── openspec/
    ├── config.yaml              # anti-assumption rules injected into every request
    └── verify-prompt.md         # cross-model assumption audit prompt

Requirements

• A llama.cpp build with llama-server (recent enough to support --n-cpu-moe and --flash-attn on).
• ~16GB VRAM (tested against an RTX 4060 Ti 16GB) and ~32GB system RAM for expert offload.
• Node.js (for the OpenSpec CLI).
• A Hugging Face account with a read token, and having accepted the license on each model's HF page while signed in (both are gated).
• A coding agent that supports a custom OpenAI-compatible base URL and model id (Cursor, Cline, OpenCode, Continue, Aider, etc.).

Install

1. Drop these files into your project root. Merge AGENTS.md and the openspec/ folder into your repo. If you already have an AGENTS.md, append the "Implementation Discipline" section from this one.

2. Install and initialize OpenSpec (if you haven't already):

   npm install -g @fission-ai/openspec@latest
   cd your-project
   openspec init          # detects your coding agent(s) and wires slash commands

   openspec init will create/overwrite its own openspec/config.yaml scaffold. After running it, re-apply the context: and rules: blocks from the config.yaml in this package (the CLI's default config won't contain them).

3. Enable the expanded profile so verify exists as a command:

   openspec config profile   # choose the profile that includes verify
   openspec update           # regenerates skills/commands for your tool

   Then open the generated verify skill for your agent (e.g. .claude/skills/openspec-verify/…, or the OpenCode/Continue/etc. equivalent) and replace its body with the contents of openspec/verify-prompt.md, keeping any YAML frontmatter the generator wrote at the top. If your version doesn't offer verify, skip this — you can run the verify prompt manually (see the workflow below). That path is more robust anyway.

4. Set your HF token and launch a model:

   export HF_TOKEN=hf_your_token_here
   make gemma4      # first run downloads + caches the GGUF (several GB)

   Edit LLAMA ?= at the top of the Makefile if your llama-server binary lives somewhere other than ./src/build/bin/llama-server.

5. Point your coding agent at the servers. Create two model profiles:

   • http://localhost:8080/v1, model id gemma4 — for explore/propose
   • http://localhost:8081/v1, model id qwen36 — for verify/apply

   No API key is needed (llama-server ignores it; put any placeholder if the agent demands one).

VRAM note: on a 16GB card you generally run one model at a time. Use make gemma4 for planning, stop it, then make qwen36 for implementation — or run them in two machines/terminals if you have the headroom. make both exists but only fits if you've verified both models load together.

Tuning the models

Both --n-cpu-moe values are starting points. Watch nvidia-smi while a model loads and runs a real long prompt:

• Loads and OOMs → raise --n-cpu-moe (more experts to system RAM).
• Loads with VRAM headroom → lower --n-cpu-moe for faster generation.
• System RAM (free -h) approaching 32GB / swapping → on qwen36, drop --no-mmap so experts page from disk instead of being fully resident.

Gemma sampling is temp 1.0 / top-k 64 / top-p 0.95 (Google's rec); Qwen is temp 1.0 / top-k 20 / top-p 0.95 (Qwen's rec) — the top-k differs, don't copy one to the other.

Example workflow

Goal: add a role+team filter to profile search in an existing codebase.

1. Explore (Gemma). Agent on gemma4. This reads the codebase first — the single best habit against building the wrong thing.

/opsx:explore add profile search filters by role and team

2. Propose (Gemma). Still on gemma4.

/opsx:propose add-profile-filters

Because of openspec/config.yaml, the generated proposal.md MUST include:

• a Current State section citing real file paths + line ranges, and
• an Assumptions section listing each dependency as a verifiable claim, e.g. "assumes searchProfiles() accepts a filters arg at src/search/profiles.ts:88", with anything unconfirmed marked UNVERIFIED.

Review the proposal yourself. The Assumptions section is your at-a-glance list of everything that could be wrong.

3. Verify (Qwen). Switch the agent to qwen36. Either:

/opsx:verify add-profile-filters

or, if you didn't install the command, paste:

Follow the instructions in openspec/verify-prompt.md for change add-profile-filters.

Qwen opens each cited file and returns a table marking every assumption VERIFIED / FALSE / UNVERIFIABLE, then a verdict. If any assumption is FALSE or UNVERIFIABLE, it says "NOT ready to apply" — go back to step 2 and fix the proposal. Only a fully CLEAR verdict proceeds.

4. Apply (Qwen). Still on qwen36.

/opsx:apply add-profile-filters

AGENTS.md keeps the agent honest here: it re-confirms assumptions against the files before editing, and if reality contradicts the spec it STOPS and reports the discrepancy instead of inventing a workaround. Tasks include writing tests for the assumed integration points and running the suite before completion.

5. Archive. Once tests pass:

/opsx:archive add-profile-filters

The change's deltas merge into openspec/specs/ (your source of truth), so the next change builds on verified reality, not stale assumptions.

Why this works

• Assumptions become explicit (config.yaml forces the section) instead of hiding in prose.
• A second, independent model fact-checks them against real code before any edit (verify-prompt.md on Qwen).
• The apply agent must stop on mismatch rather than paper over it (AGENTS.md).
• Tests execute the assumed integration points before archive.
• Specs stay current so future proposals inherit truth, not drift.

Local models will still assume more than a frontier model would — this doesn't eliminate the problem, it forces every assumption into the open and makes it survive verification and execution before you trust it.

Caveats

• OpenSpec's config/profile schema is still evolving (the OPSX rework was recent). Confirm the config.yaml keys and the verify command name against your installed version; the manual verify-prompt path works regardless.
• Model names/quant tags are matched as filename substrings by -hf repo:TAG. If a download fails listing "available GGUF files," copy the exact tag from that list (e.g. UD-Q4_K_XL, not UD-QA_K_XL).
• Both models are gated on Hugging Face — accept each license in a browser while signed in, or downloads return 401 even with a valid token.

AGENTS.md

...

block, leave it in place alongside this one. -->

Agent Rules: Implementation Discipline

When implementing against an OpenSpec change, the spec's assumptions about the current code are hypotheses, not facts. Before editing:

• Read the actual files named in the change's Current State and Assumptions sections. Confirm each assumption against what the file really contains.
• If any assumption is false — a function has a different signature, a config key doesn't exist, a module is structured differently than the spec expected — STOP. Do not code around the mismatch, do not invent a plausible substitute, and do not silently adapt the spec. Report the specific discrepancy (file, what the spec assumed, what is actually there) and wait.
• Never call a function, import a module, or reference a type you have not confirmed exists by reading its definition this session.
• When a task is done, run the tests for the integration points it touched. Report failures rather than marking the task complete.

The correct outcome when reality contradicts the spec is a flagged discrepancy, not working-looking code built on a false premise.

config.yaml

# openspec/config.yaml
# Injected into every OpenSpec planning request. This is the durable home for
# the anti-assumption guardrails: keeping it here means the discipline applies
# on every propose/explore/design without you having to ask for it each time.
#
# NOTE: OpenSpec's config schema is still evolving (the OPSX rework was recent).
# If your installed version does not recognize the `rules:` / `context:` keys,
# run `openspec init` to see the format your version expects, or move the
# `context` and `rules` prose into openspec/project.md instead — the wording
# carries over unchanged.

schema: spec-driven

context: |
  Local models run this workflow (Gemma 4 26B-A4B for propose/explore,
  Qwen3.6 35B-A3B for apply/verify). They are prone to inventing APIs,
  signatures, and config that do not exist. Every claim about the current
  implementation must be grounded in files that were actually read this
  session — never from memory or inference.

rules:
  proposal:
    - "Begin with a 'Current State' section that cites specific file paths and
       line ranges for every part of the system this change touches. If you
       cannot cite a real location, state that it is unverified — do not
       describe it as if you know it exists."
    - "End with an 'Assumptions' section listing every belief about the current
       implementation this change depends on, each written as a verifiable
       claim (e.g. 'assumes getUser() returns User | null at src/db/user.ts:42').
       Unverifiable assumptions must be labeled UNVERIFIED."
    - "Do not name a function, endpoint, type, config key, or module unless you
       have read the file it lives in during this session."
  specs:
    - "Use Given/When/Then format for every scenario."
    - "Each requirement must trace to a real integration point cited in the
       proposal's Current State. Flag any requirement that assumes behavior not
       yet confirmed against the code."
  design:
    - "For any interface, signature, or data shape you reference, quote the
       actual current definition from the codebase. If the design introduces a
       new one, mark it clearly as NEW so it is not mistaken for existing code."
  tasks:
    - "Include a task to write or update tests that exercise each integration
       point the proposal assumed, and a task to run the full test suite before
       the change is considered done."

Makefile

# Two-model local setup for the OpenSpec workflow.
#
#   gemma4  -> fast, GPU-resident. Used for /opsx:explore and /opsx:propose.
#             Served on port 8080, alias "gemma4".
#   qwen36  -> stronger, offload-bound. Used for /opsx:verify and /opsx:apply.
#             Served on port 8081, alias "qwen36".
#
# Point your coding agent at:
#   http://localhost:8080/v1  (model id: gemma4)   for propose/explore
#   http://localhost:8081/v1  (model id: qwen36)   for verify/apply
#
# Adjust LLAMA to wherever your llama-server binary lives.
# Requires: export HF_TOKEN=hf_...  and having accepted each model's license
# on its Hugging Face page while signed in.

LLAMA ?= ./src/build/bin/llama-server

.PHONY: gemma4 qwen36 both

# --- Gemma 4 26B-A4B (propose / explore) --------------------------------------
# ~15GB weights; fits your 16GB card with light expert offload.
gemma4:
	$(LLAMA) \
		-hf unsloth/gemma-4-26B-A4B-it-qat-GGUF:UD-Q4_K_XL \
		--alias gemma4 \
		--ctx-size 131072 \
		--n-gpu-layers 999 \
		--n-cpu-moe 10 \
		--flash-attn on \
		--cache-type-k q8_0 \
		--cache-type-v q8_0 \
		--temp 1.0 \
		--top-k 64 \
		--top-p 0.95 \
		--host 0.0.0.0 \
		--port 8080

# --- Qwen3.6 35B-A3B (verify / apply) -----------------------------------------
# ~22GB weights; exceeds 16GB VRAM, so heavy expert offload to system RAM.
# Tune --n-cpu-moe by watching nvidia-smi: raise if it OOMs on load,
# lower if VRAM has headroom after load. Watch `free -h` too — with weights
# in RAM you can approach your 32GB ceiling; drop --no-mmap if you start
# swapping so experts page from disk instead.
qwen36:
	$(LLAMA) \
		-hf unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_XL \
		--alias qwen36 \
		--ctx-size 131072 \
		--n-gpu-layers 999 \
		--n-cpu-moe 28 \
		--flash-attn on \
		--cache-type-k q8_0 \
		--cache-type-v q8_0 \
		--no-mmap \
		--temp 1.0 \
		--top-k 20 \
		--top-p 0.95 \
		--host 0.0.0.0 \
		--port 8081

# Convenience: launch both in the background, logging to ./logs.
# You need enough combined RAM/VRAM to hold both at once — on a 16GB card you
# generally run ONE at a time. Prefer `make gemma4` and `make qwen36` in two
# terminals unless you have verified both fit together.
both:
	@mkdir -p logs
	@echo "Launching gemma4 on :8080 and qwen36 on :8081 (logs in ./logs)"
	@$(MAKE) gemma4 > logs/gemma4.log 2>&1 &
	@$(MAKE) qwen36 > logs/qwen36.log 2>&1 &
	@echo "Started. Tail with: tail -f logs/gemma4.log logs/qwen36.log"

verify-prompts.md

Verify: Audit Change Assumptions Against Real Code

You are auditing an OpenSpec change proposal BEFORE implementation. Your only job is to determine whether each assumption the proposal makes about the current codebase is TRUE, FALSE, or UNVERIFIABLE against the actual files. You are not here to improve the plan, add features, or write code. You are a fact-checker.

Inputs

• The change folder: openspec/changes// (read proposal.md, and its Current State and Assumptions sections in particular; also design.md if present).
• The real repository files those sections reference.

Procedure

1. Extract every assumption. Collect them from the explicit "Assumptions" section AND from any factual claim in "Current State" or "Design" that asserts how the existing code behaves (a signature, return type, config key, module path, call order, data shape, existing function/endpoint/type).

2. For EACH assumption, open the actual file it refers to and read the relevant code. Do not rely on the file name sounding right, on memory, or on inference from other files. If the proposal cites a path and line, go there. If it cites no location, search the repo for the referenced symbol.

3. Classify each assumption as exactly one of:

   • VERIFIED — you read the code and it matches the claim. Cite file:line.
   • FALSE — you read the code and it contradicts the claim. Cite file:line and state what is actually there.
   • UNVERIFIABLE — the referenced code, file, or symbol does not exist, or the proposal gives too little to locate it. State what you looked for and where.

4. Do NOT fix anything. If an assumption is FALSE, do not rewrite the proposal or propose a workaround. Report it. Correcting the plan is a separate step the human decides on.

Output format

A single table, one row per assumption:

#	Assumption (short)	Verdict	Evidence (file:line + what's actually there)

Then a short verdict block:

• BLOCKERS: list every FALSE or UNVERIFIABLE assumption by number. If any exist, state clearly: "This change is NOT ready to apply — N assumptions failed."
• CLEAR: if and only if every assumption is VERIFIED, state: "All assumptions verified against current code. Ready to apply."

Rules

• Never mark an assumption VERIFIED without having read the specific code that confirms it in this session.
• When uncertain between FALSE and UNVERIFIABLE, choose UNVERIFIABLE and say why.
• Bias toward blocking. A wrongly-approved assumption costs far more than a wrongly-flagged one. If you cannot confirm it, it does not pass.
• Report discrepancies in the code's terms, not the proposal's — quote the real signature, not the assumed one.