Skip to content
Relaykit
Menu
DocsTool calling

Build

Tool calling

Let models call your functions. Define tools, run the call loop, send results back, force a tool, and handle parallel calls in OpenAI and Anthropic formats.

Updated View .md
On this page

Tool calling (function calling) lets the model ask your code to run a function, for example a database lookup or an API call, and then use the result in its answer. The model never runs anything itself: it returns the function name and JSON arguments, your code runs it, and you send the result back.

Every model with tool support in the models table accepts tools in the OpenAI format. Claude models also accept them in the Anthropic format.

The loop

  1. Send the conversation plus a list of tools, each with a name, a description and a JSON Schema for its arguments.
  2. If the reply has finish_reason: "tool_calls", run each entry in message.tool_calls.
  3. Append the assistant message, then one tool message per call with the matching tool_call_id and the result as a string.
  4. Send the request again. Repeat until the model answers without tool calls.
import json, os
from openai import OpenAI

client = OpenAI(base_url="https://api.oneaiapikey.com/v1", api_key=os.environ["RELAYKIT_API_KEY"])

def get_order_status(order_id: str) -> dict:
    return {"order_id": order_id, "status": "shipped", "eta": "2 days"}  # your real lookup

tools = [{
    "type": "function",
    "function": {
        "name": "get_order_status",
        "description": "Look up the shipping status of an order by its id.",
        "parameters": {
            "type": "object",
            "properties": {"order_id": {"type": "string", "description": "Order id, e.g. A-1042"}},
            "required": ["order_id"],
        },
    },
}]
messages = [{"role": "user", "content": "Where is my order A-1042?"}]

while True:
    reply = client.chat.completions.create(model="gpt-5.6-terra", messages=messages, tools=tools)
    msg = reply.choices[0].message
    messages.append(msg)                      # keep the assistant turn, tool_calls included
    if not msg.tool_calls:
        break
    for call in msg.tool_calls:               # may be several in one turn
        args = json.loads(call.function.arguments)
        result = get_order_status(**args)
        messages.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(result)})

print(msg.content)
import OpenAI from "openai";

const client = new OpenAI({ baseURL: "https://api.oneaiapikey.com/v1", apiKey: process.env.RELAYKIT_API_KEY });

const getOrderStatus = ({ order_id }) => ({ order_id, status: "shipped", eta: "2 days" }); // your real lookup

const tools = [{
  type: "function",
  function: {
    name: "get_order_status",
    description: "Look up the shipping status of an order by its id.",
    parameters: {
      type: "object",
      properties: { order_id: { type: "string", description: "Order id, e.g. A-1042" } },
      required: ["order_id"],
    },
  },
}];
const messages = [{ role: "user", content: "Where is my order A-1042?" }];

let msg;
while (true) {
  const reply = await client.chat.completions.create({ model: "gpt-5.6-terra", messages, tools });
  msg = reply.choices[0].message;
  messages.push(msg); // keep the assistant turn, tool_calls included
  if (!msg.tool_calls?.length) break;
  for (const call of msg.tool_calls) {
    const result = getOrderStatus(JSON.parse(call.function.arguments));
    messages.push({ role: "tool", tool_call_id: call.id, content: JSON.stringify(result) });
  }
}
console.log(msg.content);

Every round trip is a separate, separately billed request, and each one re-sends the tool definitions and the growing history. Keep tool descriptions short and cap the number of rounds in production code.

Forcing or blocking tools

tool_choice
# Always call this one tool
tool_choice={"type": "function", "function": {"name": "get_order_status"}}

# Must call some tool / may call one (default) / must not call any
tool_choice="required"   # or "auto", "none"

A forced call is also the most portable way to get structured output: define one tool whose parameters are the shape you want, and force it.

Parallel calls

Models may return several tool calls in one turn (for example, two order lookups). Run them all, in any order, and send one tool message per call before the next request. To get at most one call per turn, set parallel_tool_calls: false.

Anthropic format

On /v1/messages the shapes are Anthropic’s: tools have an input_schema, the model replies with tool_use content blocks and stop_reason: "tool_use", and you answer with tool_result blocks inside a user message.

Anthropic tool use
import os, json
import anthropic

client = anthropic.Anthropic(base_url="https://api.oneaiapikey.com", api_key=os.environ["RELAYKIT_API_KEY"])

tools = [{
    "name": "get_order_status",
    "description": "Look up the shipping status of an order by its id.",
    "input_schema": {
        "type": "object",
        "properties": {"order_id": {"type": "string"}},
        "required": ["order_id"],
    },
}]
messages = [{"role": "user", "content": "Where is my order A-1042?"}]

first = client.messages.create(model="claude-sonnet-5-5", max_tokens=1024, tools=tools, messages=messages)
if first.stop_reason == "tool_use":
    call = next(b for b in first.content if b.type == "tool_use")
    result = {"order_id": call.input["order_id"], "status": "shipped"}
    messages += [
        {"role": "assistant", "content": first.content},
        {"role": "user", "content": [
            {"type": "tool_result", "tool_use_id": call.id, "content": json.dumps(result)},
        ]},
    ]
    final = client.messages.create(model="claude-sonnet-5-5", max_tokens=1024, tools=tools, messages=messages)
    print(final.content[0].text)

When you call a Claude model in the OpenAI format, the gateway converts tools, tool_choice, parallel_tool_calls and the tool messages in both directions, so the OpenAI loop above works unchanged.

Tips

  • Write descriptions for the model, not for people: say when to use the tool and what each argument means.
  • Validate arguments before you act on them. The JSON is usually well-formed, but the values can be wrong.
  • Return errors as tool results ({"error": "order not found"}) instead of failing the request, so the model can recover.
  • When streaming, the arguments arrive in pieces. See Streaming.
  • Some reasoning models restrict tools in chat completions depending on the reasoning setting. If a model rejects your tools, the error message says why; try a lower reasoning_effort or the Responses API.

↑ ↓ moveEnter openEsc close