Inkling reasoning effort appears ineffective through Vercel AI Gateway

inkling_reasoning_effort_sweep.mjs

const prompt = `Solve this problem rigorously:

Find the smallest positive integer n such that:
- n leaves remainder 2 when divided by 5,
- n leaves remainder 3 when divided by 7,
- n leaves remainder 5 when divided by 11,
- n is divisible by 13, and
- the sum of the decimal digits of n is 23.

Show a concise derivation and verify every condition.`;

const efforts = ["low", "xhigh"];
const trials = 5;
const format = process.env.REASONING_FORMAT ?? "reasoning_effort";

const requests = efforts.flatMap((effort) =>
  Array.from({ length: trials }, (_, index) => ({ effort, trial: index + 1 })),
);

const results = await Promise.all(
  requests.map(async ({ effort, trial }) => {
    const response = await fetch(
      "https://ai-gateway.vercel.sh/v1/chat/completions",
      {
        method: "POST",
        headers: {
          Authorization: `Bearer ${process.env.AI_GATEWAY_API_KEY}`,
          "Content-Type": "application/json",
        },
        body: JSON.stringify({
          model: "thinkingmachines/inkling",
          messages: [{ role: "user", content: prompt }],
          max_tokens: 32768,
          providerOptions: { gateway: { only: ["baseten"] } },
          ...(format === "reasoning"
            ? { reasoning: { effort } }
            : { reasoning_effort: effort }),
        }),
      },
    );

    const body = await response.json();
    return {
      effort,
      trial,
      status: response.status,
      finish_reason: body.choices?.[0]?.finish_reason ?? null,
      reasoning_tokens:
        body.usage?.completion_tokens_details?.reasoning_tokens ?? null,
      completion_tokens: body.usage?.completion_tokens ?? null,
      error: body.error ?? null,
    };
  }),
);

const summary = Object.fromEntries(
  efforts.map((effort) => {
    const values = results
      .filter((result) => result.effort === effort)
      .map((result) => result.reasoning_tokens)
      .filter(Number.isFinite);
    return [
      effort,
      {
        samples: values.length,
        mean_reasoning_tokens:
          values.reduce((sum, value) => sum + value, 0) / values.length,
        min_reasoning_tokens: Math.min(...values),
        max_reasoning_tokens: Math.max(...values),
      },
    ];
  }),
);

console.log(JSON.stringify({ format, results, summary }, null, 2));

INKLING_REASONING_EFFORT_VERCEL.md

Inkling reasoning effort through Vercel AI Gateway

Neither reasoning_effort nor Vercel's reasoning.effort object shows reliable effort scaling for thinkingmachines/inkling through Vercel AI Gateway.

Thinking Machines documents reasoning_effort for its OpenAI-compatible Chat Completions API and maps low to 0.2 and xhigh to 0.99. It also warns that individual samples are stochastic, so this reproduction compares multiple samples with enough output headroom to avoid truncation.

Run

AI_GATEWAY_API_KEY=... node inkling_reasoning_effort_sweep.mjs
AI_GATEWAY_API_KEY=... REASONING_FORMAT=reasoning node inkling_reasoning_effort_sweep.mjs

Observed on July 17, 2026

Two independent five-sample runs per parameter format, with the same prompt and Vercel's Baseten route forced:

Format	Run	low mean	xhigh mean	Direction
reasoning_effort	1	4,656.8	3,675.6	reversed
reasoning_effort	2	3,545.0	4,723.8	expected
reasoning_effort	combined	4,100.9	4,199.7	+2.4%
reasoning.effort	1	3,861.4	4,410.8	expected
reasoning.effort	2	4,403.6	4,209.2	reversed
reasoning.effort	combined	4,132.5	4,310.0	+4.3%

The combined distributions overlap heavily:

• reasoning_effort low: 2,968–7,069; xhigh: 2,887–5,625
• reasoning.effort low: 3,277–5,606; xhigh: 3,465–6,086

Both formats reverse direction between repeated batches. Their combined differences are only 2.4% and 4.3%, with heavily overlapping distributions, despite comparing near-opposite effort presets. The setting appears ignored or incorrectly mapped.

References

• Thinking Machines OpenAI-compatible API: https://tinker-docs.thinkingmachines.ai/tinker/compatible-apis/openai/
• Thinking Machines effort behavior: https://tinker-docs.thinkingmachines.ai/cookbook/inkling/thinking-effort/
• Vercel Chat Completions reasoning configuration: https://vercel.com/docs/ai-gateway/sdks-and-apis/openai-chat-completions/advanced#reasoning-configuration