Tool calling

Tool calling (also called function calling) lets a model call functions that you define. You describe your tools in the OpenAI format, and LLM Router translates them for the provider that serves the request, so the same request works on OpenAI, Anthropic, Google and other models, and keeps working when the router falls back to another provider.

How it works

1. Send a chat completion with a tools array that describes your functions.
2. If the model decides to call one, the response has finish_reason: "tool_calls" and a tool_calls array on the assistant message.
3. Run the function in your own code.
4. Send a second request with the assistant message and a role: "tool" message that carries the result. The model uses it to write the final answer.

The model never runs anything itself. Your application executes every tool call.

Define tools and get a tool call

curl --request POST \
  --url https://api.inworld.ai/v1/chat/completions \
  --header "Authorization: Basic $INWORLD_API_KEY" \
  --header 'Content-Type: application/json' \
  --data '{
    "model": "openai/gpt-5",
    "messages": [
      {"role": "user", "content": "What is the weather in Paris?"}
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_weather",
          "description": "Get the current weather for a city.",
          "parameters": {
            "type": "object",
            "properties": {
              "city": {"type": "string", "description": "City name, e.g. Paris"}
            },
            "required": ["city"]
          }
        }
      }
    ]
  }'

import json
import os
from openai import OpenAI

api_key = os.environ["INWORLD_API_KEY"]
client = OpenAI(
    base_url="https://api.inworld.ai/v1",
    api_key=api_key,
    default_headers={"Authorization": f"Basic {api_key}"},
)

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a city.",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {"type": "string", "description": "City name, e.g. Paris"}
                },
                "required": ["city"],
            },
        },
    }
]
messages = [{"role": "user", "content": "What is the weather in Paris?"}]

response = client.chat.completions.create(
    model="openai/gpt-5",
    messages=messages,
    tools=tools,
)

When the model calls the tool, the assistant message looks like this:

{
  "role": "assistant",
  "content": "",
  "tool_calls": [
    {
      "id": "call_weather",
      "type": "function",
      "index": 0,
      "function": {
        "name": "get_weather",
        "arguments": "{\"city\": \"Paris\"}"
      }
    }
  ]
}

function.arguments is a JSON string, so parse it before use. content is an empty string rather than null when the model only calls tools.

Tool definition fields

Return the tool result

Append the assistant message exactly as you received it, then add one role: "tool" message per tool call with the matching tool_call_id:

assistant_message = response.choices[0].message
messages.append(assistant_message)

for tool_call in assistant_message.tool_calls:
    args = json.loads(tool_call.function.arguments)
    result = get_weather(args["city"])  # your function
    messages.append({
        "role": "tool",
        "tool_call_id": tool_call.id,
        "content": json.dumps(result),
    })

final = client.chat.completions.create(
    model="openai/gpt-5",
    messages=messages,
    tools=tools,
)
print(final.choices[0].message.content)

The tool message content can be a string or an array of text parts:

{
  "role": "tool",
  "tool_call_id": "call_weather",
  "content": [{"type": "text", "text": "The temperature in Paris is 18°C."}]
}

A tool message without a tool_call_id fails with 400.

A model can return several tool calls in one turn. Run each one and send back a tool message for every id before you ask for the next completion.

Control when tools are called

Set tool_choice to steer the model. The router translates each value to the provider's equivalent.

Streaming

With stream: true, tool calls arrive as delta.tool_calls chunks. Each entry has an index that identifies the tool call, and the arguments arrive as string fragments:

{"index": 0, "id": "call_weather", "type": "function", "function": {"name": "get_weather", "arguments": ""}}
{"index": 0, "id": "", "type": "function", "function": {"name": "", "arguments": "{\"city\":"}}
{"index": 0, "id": "", "type": "function", "function": {"name": "", "arguments": " \"Paris\"}"}}

Group the chunks by index and concatenate function.arguments. Later chunks carry empty id and name strings instead of omitting them, so keep the first non-empty value of each. The final chunk has finish_reason: "tool_calls". See Streaming for the stream format.

Differences from OpenAI

The request and response shapes follow OpenAI's chat completions API, with these differences:

• parallel_tool_calls is ignored. Models that can return several tool calls in one turn still do.
• function.strict is ignored, so arguments are not guaranteed to match your schema. Validate them before you run the tool.
• The legacy functions and function_call parameters are ignored. Use tools and tool_choice.
• Object forms of tool_choice other than the named-function form are ignored.
• Each entry in a non-streaming tool_calls array also has an index.

Tools with other features

• Voice responses: with voice responses, a turn that calls a tool produces no audio, and the answer after the tool result is spoken.
• Web search: router-run web search runs its own tool loop on the server, so you do not define or execute a search tool yourself.
• Routing: tool definitions are translated per provider, so fallbacks and conditional routing work without changes to your tools.

Next steps