pi tool-calling failure: malformed budget tag and XML tool-call loop

idfon-toolcalling-budget-tag-report.md

pi tool-calling failure: model emits XML tool syntax and loops on <budget:token_budget>

Summary

In pi coding agent 0.85.1, using Vercel AI Gateway with deepseek/deepseek-v4.1-flash at thinking level medium, the model can abandon pi's native tool-call format and emit Anthropic/XML-style tool markup as ordinary assistant text:

<function_calls>
<invoke name="bash">
<parameter name="command">...</parameter>
</invoke>
</function_calls>

It then enters a self-reinforcing loop that repeats the malformed markup and emits:

<budget:token_budget>200000</budget:token_budget>

The budget tag is not a standalone injected JSONL event. In the examined session it occurs inside an assistant message's thinking or text content, alongside the model's own discussion of the malformed tool call. The practical failure is that pi receives ordinary assistant text rather than a native toolCall block, so no tool is dispatched.

Environment

• pi coding agent: 0.85.1
• Project: idfon
• Provider: vercel-ai-gateway
• Model: deepseek/deepseek-v4.1-flash
• Thinking level: medium
• Session: 01a0abd2-5b8e-735b-b52d-e0261044f68f
• Session start: 2026-09-16T20:04:45.070Z
• Log: 2026-09-16T20-04-45-070Z_01a0abd2-5b8e-735b-b52d-e0261044f68f.jsonl

Evidence

The first token_budget occurrence is JSONL record 89. Its structure is:

{
  "type": "message",
  "message": {
    "role": "assistant",
    "content": [
      {"type": "thinking", "thinking": "...token_budget..."},
      {"type": "text", "text": "..."},
      {"type": "toolCall", "name": "read"}
    ]
  }
}

The marker is therefore not a user, system, or tool-result record in the session log. It is present in model-generated reasoning text. Later, after the model starts failing, it emits XML-style tool markup in an ordinary text block, for example:

<function_calls>
<invoke name="bash">
<parameter name="command">grep -rli token_budget ...</parameter>
</invoke>
</function_calls>

<budget:token_budget>200000</budget:token_budget>

Malformed. Retry:

The session contains valid native tool-call blocks earlier. The failure begins when the model starts writing XML markup in text instead of producing pi's structured toolCall output. The model then repeats the same attempted call and declares it malformed, without recovering.

The malformed shell command episode near the end is separate: a multiline jq command supplied by the assistant was pasted with an unterminated quote, leaving zsh at pipe quote>. That is command-example damage, not evidence of a pi parser injection.

Relevant configuration

The active pi package list did not include a pi-token-usage extension:

{
  "packages": [
    "git:github.com/DietrichGebert/ponytail",
    "npm:@dkmnx/pi-clarify",
    "npm:pi-memory",
    "npm:@latent-variable/pi-auto-continue",
    "npm:pi-mcp-adapter",
    "npm:pi-subagents",
    "npm:pi-lsp-adapter"
  ]
}

@latent-variable/pi-auto-continue only sends the literal user message continue after completed turns. It does not generate or manipulate token_budget.

Provider correlation

A previous gist documented the same model id through Vercel AI Gateway at thinking level high, where the model generated fixed-stride loops such as:

</parameter>\n</invoke>\n</parameter>\n</invoke>\n...

The same model id through OpenRouter produced clean tool-using turns in the comparison sample. This is a provider/model correlation, not yet proof of causation.

Previous report: https://gist.github.com/donpark/8a502f227849b19538d0db3499e1b5fd

Current diagnosis

The strongest explanation is a model/provider compatibility or decoding failure:

1. The model has enough accumulated context about tool markup and budget tags to enter a learned XML-tool continuation.
2. It emits that markup as text rather than pi's native structured tool-call representation.
3. The malformed markup and budget tag become part of the next context.
4. The model self-reinforces the pattern with Malformed. Retry: and repeats the pseudo-call.
5. Since no native toolCall event is emitted, pi cannot dispatch the requested tool.

The session does not establish that pi or an extension injected <budget:token_budget> into the prompt. The marker is observed in assistant-generated content. A gateway-side prompt/control-token issue remains possible, but it is not distinguishable from the session JSONL alone.

Minimal reproduction

Use a long, tool-heavy conversation at thinking level medium or high with:

provider: vercel-ai-gateway
model: deepseek/deepseek-v4.1-flash

Ask the model to inspect several files or documents and make a decision across multiple tool calls. Compare against:

• the same model through openrouter
• another reasoning-capable model through Vercel AI Gateway
• the same model with thinking disabled or lower

Record whether the assistant output contains XML tool tags, budget:token_budget, or fixed-stride repetitions, and whether the corresponding JSONL content is a native toolCall block or ordinary text/thinking.

Requested investigation

• Check whether Vercel AI Gateway transforms or injects reasoning-budget control tokens for this DeepSeek model.
• Check whether the provider adapter strips model-native budget/control tokens correctly.
• Check why XML/Anthropic tool markup is being accepted/generated despite pi's native tool schema.
• Consider a recovery guard: if a response contains repeated </invoke>, </parameter>, or <budget:token_budget> in thinking/text without native tool calls, abort the turn rather than allowing the model to loop.
• Compare the exact request and response streams between Vercel AI Gateway and OpenRouter.