Reasoning models

Reasoning models think before they answer: they write internal reasoning tokens, then the reply. That makes them better at maths, planning, debugging and multi-step tool use, and slower and more expensive per request. You decide how hard they think.

Models with reasoning

Set the effort (OpenAI format)

On chat completions, reasoning_effort sets how much the model thinks. Common values are low, medium and high; some models also accept minimal, none or xhigh. Lower effort answers faster and uses fewer tokens.

curl https://api.oneaiapikey.com/v1/chat/completions \
  -H "Authorization: Bearer $RELAYKIT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-6.1-sol",
    "reasoning_effort": "low",
    "max_completion_tokens": 4000,
    "messages": [{"role": "user", "content": "A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much is the ball?"}]
  }'

import os
from openai import OpenAI

client = OpenAI(base_url="https://api.oneaiapikey.com/v1", api_key=os.environ["RELAYKIT_API_KEY"])

reply = client.chat.completions.create(
    model="gpt-6.1-sol",
    reasoning_effort="low",
    max_completion_tokens=4000,
    messages=[{"role": "user", "content": "A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much is the ball?"}],
)
print(reply.choices[0].message.content)
print(reply.usage.completion_tokens_details.reasoning_tokens, "reasoning tokens")

import OpenAI from "openai";

const client = new OpenAI({ baseURL: "https://api.oneaiapikey.com/v1", apiKey: process.env.RELAYKIT_API_KEY });

const reply = await client.chat.completions.create({
  model: "gpt-6.1-sol",
  reasoning_effort: "low",
  max_completion_tokens: 4000,
  messages: [{ role: "user", content: "A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much is the ball?" }],
});
console.log(reply.choices[0].message.content);
console.log(reply.usage.completion_tokens_details?.reasoning_tokens, "reasoning tokens");

• Set max_completion_tokens (or max_tokens) with room for the thinking and the answer. If the limit is reached while the model is still thinking, you get finish_reason: "length" and little or no text.
• Many reasoning models accept only the default temperature and top_p. Leave them unset.
• The OpenRouter-style reasoning object ({"effort": "high"} or {"max_tokens": 4000}) is accepted as well.
• In the Responses API, use reasoning: {"effort": "low"}.

Claude models

Claude calls this extended thinking. You can turn it on in either format.

OpenAI format. Send reasoning_effort to a Claude model and the gateway turns it into Claude’s thinking settings. The thinking text comes back in message.reasoning_content (and delta.reasoning_content when streaming), separate from content.

reply = client.chat.completions.create(
    model="claude-sonnet-5-5",
    reasoning_effort="medium",          # turns on Claude's extended thinking
    max_tokens=8000,
    messages=[{"role": "user", "content": "Plan a zero-downtime migration from MySQL 5.7 to 8.0."}],
)
msg = reply.choices[0].message
print(getattr(msg, "reasoning_content", None))   # Claude's thinking, when returned
print(msg.content)                               # the answer

Anthropic format. On /v1/messages, pass Anthropic’s thinking parameter. The reply contains thinking blocks before the text block.

import anthropic, os

client = anthropic.Anthropic(base_url="https://api.oneaiapikey.com", api_key=os.environ["RELAYKIT_API_KEY"])

message = client.messages.create(
    model="claude-sonnet-5-5",
    max_tokens=16000,
    thinking={"type": "enabled", "budget_tokens": 8000},   # budget must be below max_tokens
    messages=[{"role": "user", "content": "Plan a zero-downtime migration from MySQL 5.7 to 8.0."}],
)
for block in message.content:
    if block.type == "thinking":
        print("THINKING:", block.thinking[:200], "...")
    elif block.type == "text":
        print(block.text)

In multi-turn conversations with tools, send the assistant’s thinking blocks back unchanged, as Anthropic requires. The Anthropic SDKs do this when you append message.content as it is.

What you are billed for

• Reasoning tokens are output tokens, billed at the model’s output rate, even when the reasoning text is not shown to you.
• usage.completion_tokens already includes them; usage.completion_tokens_details.reasoning_tokens shows how many there were, when the provider reports it.
• A hard question at high effort can use many times more tokens than the visible answer. Start at low and raise it only where answers get better.

Related

• Cost control: set limits per key.
• Models: output prices per model.