PydanticAI + Ollama: How to control thinking (reasoning) on/off via OpenAI-compat API

ollama_no_think_with_pydantic_ai.py

"""
PydanticAI + Ollama: Controlling Thinking (Reasoning) on/off
=============================================================

Problem:
  Ollama's OpenAI-compatible API (/v1/chat/completions) auto-enables thinking
  for thinking-capable models (like qwen3.5). There is NO `think: true/false`
  parameter on the OpenAI-compat API — that only works on Ollama's native
  API (/api/chat).

  Instead, Ollama maps the OpenAI-standard `reasoning_effort` parameter:
    - "high", "medium", "low" → thinking ON (with varying effort)
    - "none"                  → thinking OFF

  If you don't send `reasoning_effort` at all, Ollama auto-enables thinking
  for capable models — so you get thinking whether you want it or not.

Solution:
  Pass `reasoning_effort` via PydanticAI's `extra_body` mechanism.
  This can be set at three levels (from broadest to narrowest):

    1. Model level   → OpenAIChatModel(..., settings={"extra_body": {...}})
    2. Agent level   → Agent(model, model_settings={"extra_body": {...}})
    3. Per-call      → agent.run(..., model_settings={"extra_body": {...}})

  Narrower levels override broader ones.

Requirements:
  - ollama running locally (default: localhost:11434)
  - a thinking-capable model pulled (e.g. `ollama pull qwen3.5:9b`)
  - pip install pydantic-ai openai

Tested with:
  - ollama v0.9+ (2026-03)
  - pydantic-ai 0.2+
  - qwen3.5:9b

Source investigation:
  Ollama source (openai/openai.go) converts `reasoning_effort` → internal
  `Think` field. When Think is nil (no param sent), routes.go auto-sets
  Think=true for thinking-capable models. Setting `reasoning_effort: "none"`
  explicitly sets Think=false, which disables it.
"""

import asyncio

from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider

OLLAMA_BASE_URL = "http://localhost:11434/v1"
MODEL = "qwen3.5:9b"

NO_THINKING = {"extra_body": {"reasoning_effort": "none"}}


async def main():
    provider = OpenAIProvider(base_url=OLLAMA_BASE_URL, api_key="ollama")

    # ── Option A: Set at the model level (recommended) ──────────────────
    # All agents using this model inherit the setting.

    model_no_think = OpenAIChatModel(MODEL, provider=provider, settings=NO_THINKING)
    model_think = OpenAIChatModel(MODEL, provider=provider)  # thinking ON (default)

    # ── Option B: Set at the agent level ────────────────────────────────
    # agent = Agent(model_think, model_settings=NO_THINKING)

    # ── Option C: Set per-call ──────────────────────────────────────────
    # result = await agent.run("prompt", model_settings=NO_THINKING)

    prompt = "What is 2+2? Answer in one word."

    # ── Thinking ON ─────────────────────────────────────────────────────
    print("=== THINKING ON (default — no reasoning_effort) ===")
    agent = Agent(model_think)
    result = await agent.run(prompt)
    print(f"Response: {result.output}")
    print(f"Tokens:   {result.usage()}")
    # Typical output: ~200-400 tokens (model thinks internally before answering)
    response = result.all_messages()[-1]
    print("Parts:")
    for part in response.parts:
        print(f"  {type(part).__name__}: {part.content[:80]}...")
    # Expected: ThinkingPart (reasoning) + TextPart (answer)
    print()

    # ── Thinking OFF ────────────────────────────────────────────────────
    print('=== THINKING OFF (reasoning_effort="none") ===')
    agent = Agent(model_no_think)
    result = await agent.run(prompt)
    print(f"Response: {result.output}")
    print(f"Tokens:   {result.usage()}")
    # Typical output: ~2-5 tokens (direct answer, no reasoning overhead)
    response = result.all_messages()[-1]
    print("Parts:")
    for part in response.parts:
        print(f"  {type(part).__name__}: {part.content}")
    # Expected: TextPart only (no ThinkingPart)


asyncio.run(main())

Expected output for qwen3.5:9b:

=== THINKING ON (default — no reasoning_effort) ===
Response: Four
Tokens:   RunUsage(input_tokens=22, output_tokens=231, requests=1)
Parts:
  ThinkingPart: Thinking Process:

1.  **Analyze the Request:** The user is asking for the resul...
  TextPart: Four...

=== THINKING OFF (reasoning_effort="none") ===
Response: Four
Tokens:   RunUsage(input_tokens=24, output_tokens=2, requests=1)
Parts:
  TextPart: Four