Fix: mlx-lm doesn't detect LFM2's tool-call format, silently drops tools param

mlx_lm_lfm2_tool_calling_fix.md

Fixing "model does not support tool calling" for LFM2 on mlx-lm

Symptom

Running an OpenAI-compatible client (e.g. dirge or any openai SDK client) against mlx_lm.server serving LiquidAI/LFM2.5-2.6B-MLX-nvfp4 (or other LFM2 MLX conversions) with a tools=[...] payload produces this server-side warning, and the model never actually emits a tool call:

WARNING - Received tools but model does not support tool calling.
If you think this is an error, file an issue here: https://github.com/ml-explore/mlx-lm/issues

Root cause

mlx-lm (tested with 0.31.3) decides whether a model supports tool calling by sniffing the model's chat template for known marker strings, in mlx_lm/tokenizer_utils.py::_infer_tool_parser():

def _infer_tool_parser(chat_template):
    if not isinstance(chat_template, str):
        return None
    elif "<minimax:tool_call>" in chat_template:
        return "minimax_m2"
    elif "<|tool_call>" in chat_template and "<tool_call|>" in chat_template:
        return "gemma4"
    elif "<start_function_call>" in chat_template:
        return "function_gemma"
    elif "<longcat_tool_call>" in chat_template:
        return "longcat"
    elif "<arg_key>" in chat_template:
        return "glm47"
    elif "<|tool_list_start|>" in chat_template:
        return "pythonic"
    elif ("<tool_call>\\n<function=" in chat_template
          or "<tool_call>\n<function=" in chat_template):
        return "qwen3_coder"
    elif "<|tool_calls_section_begin|>" in chat_template:
        return "kimi_k2"
    elif "[TOOL_CALLS]" in chat_template:
        return "mistral"
    elif "<tool_call>" in chat_template and "tool_call.name" in chat_template:
        return "json_tools"
    return None

LFM2's chat_template.jinja renders tool calls as:

<|tool_call_start|>[func_name(arg1="value1", arg2=2)]<|tool_call_end|>

...which is exactly the format mlx_lm/tool_parsers/pythonic.py already parses (tool_call_start = "<|tool_call_start|>"). But the detector only triggers the "pythonic" branch on the string <|tool_list_start|>, which never appears in LFM2's template. Every branch falls through, tool_parser_type stays None, tokenizer.has_tool_calling is False, and server.py silently drops the tools field from the request before it ever reaches the model — so the model never even sees the tool definitions, let alone emits a call.

This is purely a gap in mlx-lm's auto-detection heuristic. The model, its chat template, and mlx-lm's own pythonic parser are all already compatible.

Fix: explicit tool_parser_type override

mlx_lm's tokenizer loader (tokenizer_utils.py::load()) checks for an explicit override before falling back to auto-detection:

tool_parser_type = tokenizer_config.get(
    "tool_parser_type", _infer_tool_parser(tokenizer.chat_template)
)

So you can force the correct parser by adding one key to the model's tokenizer_config.json:

{
  "...": "...",
  "tool_parser_type": "pythonic"
}

For a HuggingFace-cache-downloaded model, the snapshot's tokenizer_config.json is a symlink into ~/.cache/huggingface/hub/models--<org>--<model>/blobs/<hash> — edit the real blob file (resolve the symlink first, e.g. readlink -f), since the snapshot path itself is a read-only symlink.

Verification

from pathlib import Path
from mlx_lm.tokenizer_utils import load

tok = load(Path("/path/to/snapshot"))
print(tok.has_tool_calling)   # True
print(tok.tool_call_start)    # <|tool_call_start|>

Restart mlx_lm.server afterward — it caches the tokenizer at process start, so edits to tokenizer_config.json won't take effect until relaunch.

Longer-term fix

The proper fix belongs upstream in mlx-lm: add a branch to _infer_tool_parser matching <|tool_call_start|> → "pythonic", so LFM2 (and any other model using this exact marker convention) is auto-detected without a manual config patch. Consider filing/upvoting an issue at https://github.com/ml-explore/mlx-lm/issues.

---

Diagnosed and documented by Claude (Sonnet 5), via Claude Code.