Realtime API (WebRTC)

Overview

The Realtime WebRTC API has three parts:

1. Signaling endpoint — Used to negotiate the WebRTC connection via SDP offer/answer exchange.

POST https://api.inworld.ai/v1/realtime/calls

2. ICE servers — STUN/TURN server configurations for NAT traversal and reliable connectivity.

GET https://api.inworld.ai/v1/realtime/ice-servers

3. Data channel events — Once the WebRTC peer connection is established, all Realtime events flow over a data channel named oai-events.

Data Channel: oai-events

Examples

Get started quickly with these reference implementations: JavaScript and Python.

Authentication

Use your API key for authentication. See Authentication for details.

Authorization: Bearer <API_KEY>

Signaling Endpoints

Create Call

Creates a WebRTC call by posting an SDP offer and optional session configuration. When provided, the proxy forwards session upstream as an initial session.update after the WebSocket connection succeeds and before returning the SDP answer. A failure to send that update fails call creation.

Request body:

Response body:

{
  "sdp": "v=0\r\no=- 4611731400430051336 2 IN IP4 127.0.0.1\r\n...",
  "session": {
    "model": "llama-3.3-70b-versatile",
    "instructions": "You are a helpful assistant.",
    "output_modalities": ["audio", "text"],
    "audio": {
      "input": {
        "transcription": {
          "model": "inworld/inworld-stt-1"
        },
        "turn_detection": {
          "type": "semantic_vad",
          "eagerness": "medium"
        }
      },
      "output": {
        "model": "inworld-tts-2",
        "voice": "Dennis",
        "speed": 1.0
      }
    }
  }
}

{
  "id": "call_abc123",
  "sdp": "v=0\r\no=- 1234567890 2 IN IP4 0.0.0.0\r\n...",
  "ice_servers": [
    {
      "urls": ["stun:stun.l.google.com:19302"]
    },
    {
      "urls": ["turn:turn.example.com:3478"],
      "username": "<TURN_USERNAME>",
      "credential": "<TURN_CREDENTIAL>"
    }
  ]
}

Get ICE Servers

Returns STUN and TURN server configurations for WebRTC connectivity. Use these ICE servers when creating the RTCPeerConnection to ensure reliable connections across NATs and firewalls.

Response body:

{
  "ice_servers": [
    {
      "urls": [
        "stun:stun.l.google.com:19302",
        "stun:stun1.l.google.com:19302"
      ]
    },
    {
      "urls": [
        "turn:34.41.153.85:3478",
        "turn:34.41.153.85:3479?transport=tcp"
      ],
      "username": "1772761055:6e3fa6ea-2ed0-4306-971e-aa5092cb3736",
      "credential": "6eBPxGW2nsktPFzjjbJSF5PK8ow="
    }
  ]
}

Data Channel Events

Once the WebRTC connection is established, events are exchanged as JSON messages over the oai-events data channel. The event protocol is the same as the Realtime WebSocket API.

Client Events

Events sent from the client to the server.

session.update

Update the session configuration. The server responds with a session.updated event.

{
  "type": "session.update",
  "session": {
    "instructions": "You are a friendly voice assistant.",
    "audio": {
      "input": {
        "transcription": {
          "model": "inworld/inworld-stt-1"
        },
        "turn_detection": {
          "type": "semantic_vad",
          "eagerness": "medium",
          "create_response": true,
          "interrupt_response": true
        }
      },
      "output": {
        "model": "inworld-tts-2",
        "voice": "Dennis",
        "speed": 1.0
      }
    }
  }
}

conversation.item.create

Add a conversation item (message, function call result, etc.).

{
  "type": "conversation.item.create",
  "item": {
    "type": "message",
    "role": "user",
    "content": [
      { "type": "input_text", "text": "Hello, how are you?" }
    ]
  }
}

conversation.item.truncate

Truncate an assistant message's audio.

conversation.item.delete

Delete a conversation item by ID.

conversation.item.retrieve

Retrieve a conversation item by ID.

response.create

Trigger a model response. The server streams back response events.

{
  "type": "response.create",
  "response": {
    "output_modalities": ["audio", "text"],
    "instructions": "Respond in a cheerful tone."
  }
}

response.cancel

Cancel an in-progress response.

input_audio_buffer.append

Append audio bytes to the input buffer.

input_audio_buffer.commit

Commit the buffered audio as a user message.

input_audio_buffer.clear

Discard all audio in the input buffer.

output_audio_buffer.clear

Clear the server's output audio buffer, stopping playback.

Server Events

Events emitted by the server to the client.

session.created

session.updated

Confirms a session.update was applied.

error

Indicates an error occurred.

conversation.item.added

A new item was added to the conversation.

conversation.item.done

An item finished being populated.

Other conversation item events

conversation.item.input_audio_transcription.delta

Streaming partial transcription for user audio.

conversation.item.input_audio_transcription.completed

Final transcription for a user audio item.

response.created

A new response was created. Contains the full response object in its initial state.

response.done

The response finished. Contains the completed response object with final status and output.

response.output_item.added

An output item was added to the response.

response.output_item.done

An output item finished.

response.content_part.added

A content part was added to an output item.

response.content_part.done

A content part finished.

response.output_text.delta

Streaming text chunk from the model.

response.output_text.done

Text output finished.

response.output_audio_transcript.delta

Streaming transcript for generated audio.

response.output_audio_transcript.done

Final transcript for generated audio.

response.output_audio.delta

Streaming audio alignment data for generated speech. Over WebRTC, audio travels on the RTP media track; this data-channel event carries only timestamp_info (the delta field is empty). Present only when providerData.tts.timestamp_type is set.

response.output_audio.done

Audio output for a content part finished.

response.function_call_arguments.delta

Streaming function call arguments.

response.function_call_arguments.done

Function call arguments finished.

input_audio_buffer.speech_started

Voice activity detected — user started speaking.

input_audio_buffer.speech_stopped

Voice activity ended — user stopped speaking.

input_audio_buffer.committed

Buffered audio was committed as a conversation item.

input_audio_buffer.timeout_triggered

Idle timeout fired on the input buffer (server VAD only — gated by turn_detection.idle_timeout_ms).

input_audio_buffer.turn_suggestion

Server VAD smart-turn detector predicts an end-of-turn boundary. Use this signal to drive low-latency UI cues or to pre-warm a response without waiting for the final speech_stopped commit. May be followed by input_audio_buffer.turn_suggestion_revoked if the user resumes speaking.

input_audio_buffer.turn_suggestion_revoked

Emitted when the user resumes speaking after a previous turn_suggestion. Pairs with the most recent turn_suggestion sharing the same utterance_index.

Other audio buffer events

response.backchannel.audio.delta

Streaming PCM audio chunk for a low-latency back-channel interjection (e.g. "uh-huh", "right") emitted while the user is mid-utterance. Out-of-band from the main response stream — use backchannel_id to group chunks belonging to the same interjection.

response.backchannel.audio.done

All audio for a back-channel interjection has been streamed. No teardown required — playback queues until exhausted.

response.backchannel.skipped

An evaluation tick chose not to fire a back-channel. Useful for client-side telemetry; clients that don't care can ignore this event.

rate_limits.updated

Reports current rate limit state.

Schemas

Session object

The session object configures model behavior, audio settings, tools, and more. It appears in signaling requests, session.update, session.created, and session.updated events.

AudioConfig

AudioFormat

TurnDetection

Turn detection has two modes, selected by the type field.

Server VAD (type: "server_vad"):

Semantic VAD (type: "semantic_vad"):

ConversationItem

ContentPart

ResponseConfig

Per-response overrides for session defaults.

Tool

ToolChoiceTarget

Specifies tool choice behavior. The server always returns this as an object.