Tool calling

Tool calling (also called function calling) lets your agent fetch live data or trigger actions mid-conversation. You register tools on the session, the model decides when to call one, your code runs it, and the model uses the result to continue the conversation.

The flow follows the OpenAI Realtime event shapes, so the wire names keep OpenAI's function wording: tools have type: 'function', calls arrive as response.function_call_arguments.done, and you return results as function_call_output items.

How it works

1. Register tools in session.tools with session.update.
2. When the model calls a tool, you receive response.function_call_arguments.done with the call_id, the function name, and the serialized arguments.
3. Run your function and add a function_call_output item with the same call_id.
4. The model incorporates the result and continues the conversation. By default Inworld starts this follow-up response for you. See Control tool continuation.

Register a tool

Define each tool with a name, a description, and a JSON Schema for its parameters. session.update accepts partial updates, so you can add or change tools during the conversation.

ws.send(JSON.stringify({
  type: 'session.update',
  session: {
    type: 'realtime',
    tools: [{
      type: 'function',
      name: 'get_horoscope',
      description: 'Get the horoscope for a zodiac sign',
      parameters: {
        type: 'object',
        properties: {
          sign: {
            type: 'string',
            description: 'Zodiac sign, e.g. Aries, Taurus'
          }
        },
        required: ['sign']
      }
    }],
    tool_choice: 'auto'
  }
}));

Handle a tool call

Listen for response.function_call_arguments.done, run your logic, and send the result back as a function_call_output item:

ws.on('message', (buffer) => {
  const event = JSON.parse(buffer.toString());

  if (event.type === 'response.function_call_arguments.done') {
    const { call_id, name, arguments: argsJson } = event;
    const args = JSON.parse(argsJson);

    // Run your business logic
    let result;
    if (name === 'get_horoscope') {
      result = fetchHoroscope(args.sign);
    }

    // Send the tool result back
    ws.send(JSON.stringify({
      type: 'conversation.item.create',
      item: {
        type: 'function_call_output',
        call_id,
        output: JSON.stringify(result)
      }
    }));
  }
});

After the result is added, the model continues the conversation: it speaks the answer aloud if output_modalities includes audio, or streams text deltas.

You can register multiple tools and the model calls them as needed. Each call arrives as a separate response.function_call_arguments.done event with its own call_id. To read arguments as they are generated, listen for response.function_call_arguments.delta.

Control tool continuation

providerData.auto_tool_response controls who starts the next response after you add a function_call_output item.

We recommend keeping the default, since it ensures tool calls are responded to as soon as possible. Set it to false when you are migrating an OpenAI client that controls continuation itself, or when you want consecutive tool calls followed by only one response:

ws.send(JSON.stringify({
  type: 'session.update',
  session: {
    type: 'realtime',
    providerData: {
      auto_tool_response: false
    }
  }
}));

// Later, after adding the function_call_output item:
ws.send(JSON.stringify({ type: 'response.create' }));

The field is hot-swappable. Omitting it from a later partial update preserves its current value. See Tool continuation in the extensions reference.

Validation

When your client creates the items itself, a function_call item requires a non-empty name, and a function_call_output item requires a non-empty call_id. Omitted arguments and output are valid and normalize to empty strings, so argumentless calls and tool results without content are accepted.

Next steps