Prompting Small Context Models

prompting_small_context_models.md

Prompting Small Context Models (≈4k tokens)

Core Principle

Treat the model like a stateless function:

• You control the inputs
• It returns a result
• It does not reliably “remember” beyond what you provide

---

1. Scope Tightly

Break problems into small units:

• One function
• One handler + test
• One query

Avoid:

• Dumping entire codebases or long documents

---

2. Provide Minimum Sufficient Context

Include:

• Relevant code
• Inputs/outputs
• Errors

Exclude:

• Boilerplate
• Unrelated files
• Repetition

---

3. Frame the Task Explicitly

Guide attention:

• “Focus on error handling”
• “Only analyze the database interaction”

---

4. Iterate in Steps

Don’t aim for one-shot answers.

Pattern:

1. Analyze
2. Fix
3. Refactor
4. Validate

---

5. Summarize Between Steps

Compress context as you go:

• Restate key facts
• Drop unnecessary detail

---

6. Externalize Memory

Keep important context outside the model:

• Notes
• Architecture summaries
• Relationships between components

Then re-inject only what’s needed.

---

7. Reference Instead of Repeating

Avoid re-pasting everything:

• “Using the handler above…”
• Only include changed or relevant parts

---

8. Ask for Transformations

Best use cases:

• Refactor this code
• Convert to table-driven tests
• Simplify logic

Avoid vague, open-ended prompts.

---

9. Control Output Size

Be explicit:

• “Be concise”
• “Only return the function”
• “Show a diff”

---

10. Think in Pipelines

Chain multiple small prompts:

• Each step = focused input + output
• Build solutions incrementally

---

Mental Model

Don’t ask:

“How do I fit everything into the model?”

Ask:

“What is the smallest useful slice of this problem?”