LLM Wiki vs Markdown-Native Knowledge: Architecture Comparison

Karpathy's LLM Wiki gist proposes that LLMs should maintain a persistent wiki layer between you and your raw sources. It's an elegant pattern. But after implementing it, I found a lighter approach that keeps the best part — knowledge compounding — without the maintenance burden.

The Two Architectures

Karpathy's LLM Wiki (3-Layer)

┌─────────────────────────────────────────────┐
│  Schema (CLAUDE.md)                         │
│  ├── Wiki structure rules                   │
│  ├── Naming conventions                     │
│  └── Workflow definitions                   │
├─────────────────────────────────────────────┤
│  Wiki (LLM-maintained markdown)             │
│  ├── Entity pages, concept summaries        │
│  ├── Cross-references between pages         │
│  ├── index.md (content catalog)             │
│  └── log.md (audit trail)                   │
│  ⚡ LLM reads + writes this layer           │
├─────────────────────────────────────────────┤
│  Raw Sources (immutable)                    │
│  ├── Articles, papers, docs, images         │
│  └── LLM reads but never modifies           │
└─────────────────────────────────────────────┘

Ingest:  LLM reads source → updates 10-15 wiki pages → appends to log
Query:   LLM searches wiki → answers → optionally creates new wiki page
Lint:    Periodic LLM pass checking contradictions, orphans, stale claims

Markdown-Native Knowledge (What I Built)

┌─────────────────────────────────────────────┐
│  Syntheses (compounding layer)              │
│  ├── Persisted query answers                │
│  ├── Cross-cutting insights                 │
│  └── Auto-indexed on next reindex cycle     │
│  ⚡ LLM writes occasionally, never maintains │
├─────────────────────────────────────────────┤
│  RAG Index (auto-generated, deterministic)  │
│  ├── index.md (domain-organized catalog)    │
│  ├── BM25 term frequency index              │
│  ├── manifest.yaml (machine catalog)        │
│  └── Zero LLM cost — pure metadata          │
├─────────────────────────────────────────────┤
│  Raw Sources (markdown-first)               │
│  ├── Notes, docs, ADRs, configs             │
│  ├── All markdown with YAML frontmatter     │
│  └── Git-versioned, grep-searchable         │
└─────────────────────────────────────────────┘

Ingest:  Add markdown file → next reindex picks it up → BM25 + manifest updated
Query:   BM25 + ripgrep hybrid → reciprocal rank fusion → LLM reranks top-K
Compound: Valuable answer? save-synthesis → indexed next cycle

Why I Dropped the Wiki Layer

Concern	LLM Wiki	Markdown-Native
LLM dependency	Every ingest, lint, and update requires LLM	Index generation is deterministic — zero LLM calls
Hallucination risk	Wiki pages can silently contradict sources	No generated layer to diverge from truth
Token cost per ingest	10-50x (read source + update wiki pages)	~0 (add file, reindex is metadata-only)
Offline capability	Ingest/lint blocked without API access	Full read/write/search works offline
Model portability	Wiki consistency tied to one model	Model-agnostic — any LLM can query the index
Vault bloat	Wiki pages grow proportionally to sources	Index is compact (~3,600 lines for 8,000+ entries)
Auditability	log.md tracks changes, wiki edits opaque	Git history tracks every character change
Human readability	Readable but LLM-authored	Human-authored markdown, browseable in Obsidian

The 2 Functions I Kept

1. Content-Oriented index.md

Auto-generated at the end of every reindex cycle. Organized by domain, not alphabetically. An LLM reads this first to navigate the knowledge base, then drills into specific files.

# Knowledge Index
# 8,247 entries across 12 categories
# Generated: 2026-04-05

## Adaptive Engine (2,891 entries)

### Auto-Commands
- **auto-lint** — ESLint auto-fix and guided repair across codebase
- **auto-rag-reindex** — Rebuild RAG indexes for all skill data dirs
- **auto-tech-debt** — Identify and prioritize technical debt items

## Brain (847 entries)

### Knowledge Management
- **rag-hybrid-search** — BM25 + ripgrep fusion with RRF ranking
- **save-synthesis** — Persist valuable query answers as vault notes

Properties:

Generated from existing manifest metadata — no LLM involved
Regenerated every reindex — never goes stale
~3,600 lines covering 8,000+ entries
Works in Obsidian, VS Code, any markdown viewer

2. save-synthesis

When a search produces a genuinely valuable answer, persist it as a markdown note:

def save_synthesis(query: str, synthesis: str, sources: list[str] = [], tags: list[str] = []):
    """Persist a valuable query answer as a markdown note."""
    slug = slugify(query)[:60]
    date = datetime.now().strftime("%Y-%m-%d")
    path = vault_dir / "knowledge" / "syntheses" / f"{date}-{slug}.md"

    frontmatter = {
        "type": "synthesis",
        "query": query,
        "date": date,
        "sources": sources,
        "tags": tags,
    }
    write_frontmatter(path, frontmatter=frontmatter, body=synthesis)
    # Indexed automatically on next reindex cycle — no extra LLM call needed

The synthesis becomes searchable in future queries via BM25 + ripgrep. Over time, syntheses accumulate as a lightweight knowledge layer — compounding without wiki infrastructure.

The Search Pipeline

The retrieval system that replaces wiki navigation:

Query → ┬─ BM25 (term frequency matching)
        ├─ ripgrep (exact pattern matching)
        └─ Reciprocal Rank Fusion (merge results)
              │
              ▼
        Top-K candidates
              │
              ▼
        LLM reranks by relevance
              │
              ▼
        Answer (optionally → save-synthesis)

BM25 handles semantic similarity. ripgrep handles exact matches (function names, error codes, config keys). RRF merges both ranked lists without needing to normalize scores.

The LLM only appears at the end — for reranking and answering. Never for maintenance.

Decision Framework

LLM Wiki works best when:

Knowledge domain is narrow and well-defined
You need entity-relationship graphs the LLM maintains
Token budget is generous (enterprise scale)
You want the LLM to proactively surface connections
Single-model deployment, no switching

Markdown-Native works best when:

Knowledge base is large (1,000+ sources)
Offline/local-first capability matters
You switch between models (Claude, GPT, Ollama, etc.)
You want git-versioned, auditable knowledge
Token efficiency matters
You value grep-searchability and human editability

Hybrid approach:

Small, high-value LLM wiki for core domain concepts
Everything else as indexed markdown
Wiki references markdown sources, not the other way around

The Core Insight

Karpathy's key contribution isn't the wiki architecture — it's the principle that knowledge should compound.

Every query that produces insight should leave a trace. That trace should be findable next time someone asks a related question.

You don't need an LLM-maintained wiki to achieve this. You need:

A navigable index (auto-generated from metadata)
A way to persist valuable answers (save-synthesis)
A search system that finds both sources and past syntheses (BM25 + ripgrep)

The wiki is one implementation of compounding knowledge. Connected markdown is another.

Built as part of Augur — an edge-first knowledge OS where your data stays as markdown you own.

gsannikov/llm-wiki-vs-markdown-native-knowledge.md

Select an option

No results found