Realtime TTS-2 is live. Built for realtime conversation that feels human. Read the Realtime TTS-2 announcement

Realtime API

Realtime API (WebRTC)

Technical specification for Realtime WebRTC signaling and event schemas, including SDP call creation and data channel event contracts.

Overview

The Realtime WebRTC API has these parts:

  1. Signaling endpoint — Negotiate the WebRTC connection by exchanging an SDP offer and answer.
POST https://api.inworld.ai/v1/realtime/calls
  1. ICE servers — STUN/TURN server settings for reliable connections across NATs.
GET https://api.inworld.ai/v1/realtime/ice-servers
  1. Data channel events — Once the WebRTC peer connection is established, all Realtime events use the oai-events data channel.
Data Channel: oai-events
  1. Call control (beta) — End a call, or attach a server WebSocket to its events. See Call Control.
POST https://api.inworld.ai/v1/realtime/calls/{call_id}/hangup
wss://api.inworld.ai/v1/realtime?call_id={call_id}

Examples

Start with these reference implementations: JavaScript and Python.

Authentication

Browser and mobile clients use a session token created on your backend. Send the token as a Bearer credential:

Authorization: Bearer <session-token>

Server-side clients can use the complete Base64 API credential copied from Portal or the CLI:

text
Authorization: Basic <INWORLD_API_KEY>
Keep API keys on your server. Never send them to browser or mobile code. See Security best practices.

See Authentication for credential formats and header details.


Signaling Endpoints

Create Call

Create a WebRTC call by posting an SDP offer and optional session configuration. If you provide session, the proxy sends it upstream as an initial session.update after the WebSocket connection succeeds and before returning the SDP answer. If that update cannot be sent, call creation fails.

Content-TypeRequestResponse
application/json{ "sdp": "...", "session": { ... } }JSON call object with id, sdp, and ice_servers.
multipart/form-datasdp field plus optional JSON-string session fieldRaw SDP answer (application/sdp).
application/sdpRaw SDP offerRaw SDP answer (application/sdp).

A successful request returns 201 Created with a Location: /v1/realtime/calls/{call_id} header for every content type. The call ID in it is a UUID, the same value as the JSON response's id. Use it for call control.

Request body:

FieldTypeRequiredDescription
sdpstringYesSDP offer generated by the client RTCPeerConnection.
sessionSessionNoInitial session configuration.

Response body:

FieldTypeDescription
idstringServer-assigned call identifier (a UUID). Also in the Location header.
sdpstringSDP answer returned by the server.
ice_serversobject[]Array of ICE server configurations (same schema as the Get ICE Servers response).
Example
Request
{
  "sdp": "v=0\r\no=- 4611731400430051336 2 IN IP4 127.0.0.1\r\n...",
  "session": {
    "model": "llama-3.3-70b-versatile",
    "instructions": "You are a helpful assistant.",
    "output_modalities": ["audio", "text"],
    "audio": {
      "input": {
        "transcription": {
          "model": "inworld/inworld-stt-1"
        },
        "turn_detection": {
          "type": "semantic_vad",
          "eagerness": "medium"
        }
      },
      "output": {
        "model": "inworld-tts-2",
        "voice": "Dennis",
        "speed": 1.0
      }
    }
  }
}
Response
{
  "id": "3f9a1c2e-5b7d-4e8f-9a0b-1c2d3e4f5a6b",
  "sdp": "v=0\r\no=- 1234567890 2 IN IP4 0.0.0.0\r\n...",
  "ice_servers": [
    {
      "urls": ["stun:stun.l.google.com:19302"]
    },
    {
      "urls": ["turn:turn.example.com:3478"],
      "username": "<TURN_USERNAME>",
      "credential": "<TURN_CREDENTIAL>"
    }
  ]
}

Get ICE Servers

Get STUN and TURN server settings for WebRTC. Use these ICE servers when creating the RTCPeerConnection for reliable connections across NATs and firewalls.

Response body:

FieldTypeDescription
ice_serversobject[]Array of ICE server configurations.
ice_servers[].urlsstring[]STUN or TURN server URLs.
ice_servers[].usernamestringTURN credential username (only for TURN servers).
ice_servers[].credentialstringTURN credential (only for TURN servers).
Example
Response
{
  "ice_servers": [
    {
      "urls": [
        "stun:stun.l.google.com:19302",
        "stun:stun1.l.google.com:19302"
      ]
    },
    {
      "urls": [
        "turn:34.41.153.85:3478",
        "turn:34.41.153.85:3479?transport=tcp"
      ],
      "username": "1772761055:6e3fa6ea-2ed0-4306-971e-aa5092cb3736",
      "credential": "6eBPxGW2nsktPFzjjbJSF5PK8ow="
    }
  ]
}

Call Control

Call control is in beta, its behavior may still change. See WebRTC call control for when to use it and a full example.

Call control works only with WebRTC calls. Both endpoints take the call ID from Create Call and authenticate with your API key (Authorization: Basic <INWORLD_API_KEY>).

Hang Up Call

End a WebRTC call. The request has no body. About 2 seconds after the response, Inworld closes the call's data channel, its peer connection and every attached sideband.

Path parameters:

FieldTypeRequiredDescription
call_idstringYesCall ID from the Location header or id of Create Call.

Responses:

StatusDescription
200The call has started to end. Empty body.
401No credential, or the credential is invalid.
404No live call with this ID in your workspace: it already ended (including a second hangup), or the ID is unknown or malformed. Plain response, no JSON body.
429Rate limited. The response may include Retry-After. See Handling rate-limited requests.
Example
bash
curl -X POST "https://api.inworld.ai/v1/realtime/calls/$CALL_ID/hangup" \
  -H "Authorization: Basic $INWORLD_API_KEY"

Sideband WebSocket

wss://api.inworld.ai/v1/realtime?call_id={call_id}

Attach a server WebSocket to a live WebRTC call. It receives the same JSON events as the data channel.

Query parameters:

FieldTypeRequiredDescription
call_idstringYesCall ID from Create Call.

Authentication: send Authorization: Basic <INWORLD_API_KEY>. This is the recommended form. For a server-side WebSocket library that cannot set headers, send the WebSocket subprotocol basic_<INWORLD_API_KEY> instead, with any trailing = padding removed from the key. The server selects that subprotocol in its answer. The subprotocol form works only when the key contains no /, which is not a valid subprotocol character. Never use it from a browser: the API key stays on your server.

Handshake errors (before the upgrade):

StatusDescription
401No credential, or the credential is invalid.
404call_id is missing, or names no live call in your workspace.
429Rate limited. The limit applies to the attach, not to messages on an open sideband. The response may include Retry-After. See Handling rate-limited requests.

Behavior:

  • Server events: the sideband receives every server event the call's data channel receives, in the same order, including error events. Nothing is sent on attach (no session.created). Response audio travels on the media track, so response.output_audio.delta reaches the sideband only as an alignment event with an empty delta, when providerData.tts.timestamp_type is set. When back-channel is enabled, response.backchannel.audio.delta events, which carry audio, are sent too.
  • Client events: client events sent on the sideband apply to the call exactly as on the data channel. input_audio_buffer.append is refused with an error event, code sideband_audio_not_supported. The error is sent on the call, so the client's data channel and every sideband receive it. A message that is not valid JSON, or is not a known client event, is ignored without an error event, as on the data channel.
  • Several sidebands: a call can have any number of sidebands. Each receives every event.
  • Limits: only the attach is rate limited; events are never throttled. A sideband does not count toward your concurrent session limit and is not billed separately. A sideband that falls too far behind on reading events is closed; the call continues.
  • Closing: closing a sideband never ends the call. When the call ends, Inworld closes every sideband about 2 seconds later without a close frame, so clients see an abnormal closure (1006).

Data Channel Events

Once the WebRTC connection is established, send and receive events as JSON messages over the oai-events data channel. It uses the same event protocol as the Realtime WebSocket API.

Client Events

Events sent from the client to the server.

session.update

Update the session configuration. The server responds with a session.updated event.

FieldTypeRequiredDescription
typesession.updateYesEvent type.
event_idstringNoOptional client-generated event ID.
sessionSessionYesSession configuration to apply.
Example
json
{
  "type": "session.update",
  "session": {
    "instructions": "You are a friendly voice assistant.",
    "audio": {
      "input": {
        "transcription": {
          "model": "inworld/inworld-stt-1"
        },
        "turn_detection": {
          "type": "semantic_vad",
          "eagerness": "medium",
          "create_response": true,
          "interrupt_response": true
        }
      },
      "output": {
        "model": "inworld-tts-2",
        "voice": "Dennis",
        "speed": 1.0
      }
    }
  }
}

conversation.item.create

Add a conversation item (message, tool result, etc.).

FieldTypeRequiredDescription
typeconversation.item.createYesEvent type.
event_idstringNoOptional client-generated event ID.
previous_item_idstringNoOmit to append, use "root" to prepend, or provide an existing item ID to insert immediately after it. An unknown ID returns item_not_found without changing history.
itemConversationItemYesThe item to add.
Example
json
{
  "type": "conversation.item.create",
  "item": {
    "type": "message",
    "role": "user",
    "content": [
      { "type": "input_text", "text": "Hello, how are you?" }
    ]
  }
}

conversation.item.truncate

Truncate an assistant message's audio.

FieldTypeRequiredDescription
typeconversation.item.truncateYesEvent type.
event_idstringNoOptional client-generated event ID.
item_idstringYesThe ID of the assistant message item to truncate.
content_indexintegerYesIndex of the content part to truncate.
audio_end_msintegerYesMillisecond offset to truncate the audio at.

conversation.item.delete

Delete a conversation item by ID.

FieldTypeRequiredDescription
typeconversation.item.deleteYesEvent type.
event_idstringNoOptional client-generated event ID.
item_idstringYesThe ID of the conversation item to delete.

conversation.item.retrieve

Retrieve a conversation item by ID.

FieldTypeRequiredDescription
typeconversation.item.retrieveYesEvent type.
event_idstringNoOptional client-generated event ID.
item_idstringYesThe ID of the conversation item to retrieve.

response.create

Trigger a model response. The server streams back response events.

FieldTypeRequiredDescription
typeresponse.createYesEvent type.
event_idstringNoOptional client-generated event ID.
responseResponseConfigNoOverride session defaults for this response.
Example
json
{
  "type": "response.create",
  "response": {
    "output_modalities": ["audio", "text"],
    "instructions": "Respond in a cheerful tone."
  }
}

response.cancel

Cancel an in-progress response.

FieldTypeRequiredDescription
typeresponse.cancelYesEvent type.
event_idstringNoOptional client-generated event ID.
response_idstringNoCancel a specific response by ID. If omitted, cancels the active response.

input_audio_buffer.append

Append audio bytes to the input buffer.

FieldTypeRequiredDescription
typeinput_audio_buffer.appendYesEvent type.
event_idstringNoOptional client-generated event ID.
audiostringYesBase64-encoded audio chunk (~100–200ms) matching the configured input format.

input_audio_buffer.commit

Commit the buffered audio as a user message.

FieldTypeRequired
typeinput_audio_buffer.commitYes
event_idstringNo

input_audio_buffer.clear

Discard all audio in the input buffer.

FieldTypeRequired
typeinput_audio_buffer.clearYes
event_idstringNo

output_audio_buffer.clear

Clear the server's output audio buffer, stopping playback.

FieldTypeRequired
typeoutput_audio_buffer.clearYes
event_idstringNo

Server Events

Events emitted by the server to the client.

session.created

This event arrives over the data channel once it opens and contains the session's default configuration. You don't need to wait for it. Send session.update to configure the session as soon as the data channel opens.

FieldTypeDescription
typesession.createdEvent type.
event_idstringServer-generated event ID.
sessionSessionCurrent session configuration.

session.updated

Confirms a session.update was applied.

FieldTypeDescription
typesession.updatedEvent type.
event_idstringServer-generated event ID.
sessionSessionUpdated session configuration.

error

Indicates an error occurred.

FieldTypeDescription
typeerrorEvent type.
event_idstringServer-generated event ID.
error.typestringError category.
error.codestringError code.
error.messagestringHuman-readable error description.
error.paramstringRelated parameter, if applicable.
error.event_idstringThe client event ID that caused the error, if applicable.

conversation.item.added

A new item was added to the conversation.

FieldTypeDescription
typeconversation.item.addedEvent type.
event_idstringServer-generated event ID.
previous_item_idstringThe ID of the preceding conversation item, or null.
itemConversationItemThe item that was added.

conversation.item.done

An item finished being populated.

FieldTypeDescription
typeconversation.item.doneEvent type.
event_idstringServer-generated event ID.
previous_item_idstringThe ID of the preceding conversation item, or null.
itemConversationItemThe completed item.

Other conversation item events

Event typeDescription
conversation.item.retrievedResponse to conversation.item.retrieve.
conversation.item.deletedAn item was deleted from the conversation.
conversation.item.truncatedAn assistant audio item was truncated.

conversation.item.input_audio_transcription.delta

Streaming partial transcription for user audio.

FieldTypeDescription
typeconversation.item.input_audio_transcription.deltaEvent type.
event_idstringServer-generated event ID.
item_idstringThe conversation item being transcribed.
content_indexintegerIndex of the content part being transcribed.
deltastringPartial transcription text.

conversation.item.input_audio_transcription.completed

Final transcription for a user audio item.

FieldTypeDescription
typeconversation.item.input_audio_transcription.completedEvent type.
event_idstringServer-generated event ID.
item_idstringThe conversation item that was transcribed.
content_indexintegerIndex of the content part that was transcribed.
transcriptstringComplete transcription text.

response.created

A new response was created. Contains the full response object in its initial state.

FieldTypeDescription
typeresponse.createdEvent type.
event_idstringServer-generated event ID.
response.idstringResponse identifier.
response.objectrealtime.responseObject type.
response.statusstring"in_progress".
response.status_detailsobject | nullStatus details, if any.
response.outputarrayOutput items (empty at creation).
response.conversation_idstringConversation this response belongs to.
response.output_modalitiesstring[]"text", "audio", or both.
response.max_output_tokensinteger | "inf"Token limit for this response.
response.audioobjectAudio output config echoed from session.
response.usageobject | nullToken usage (populated in response.done).
response.metadataobjectClient metadata from response.create, echoed unchanged. Omitted when not supplied.

response.done

The response finished. Contains the completed response object with final status and output.

FieldTypeDescription
typeresponse.doneEvent type.
event_idstringServer-generated event ID.
response.idstringResponse identifier.
response.objectrealtime.responseObject type.
response.statusstring"completed", "cancelled", or "failed".
response.status_detailsobjectStatus details. type matches status. For cancelled: includes reason (e.g., "client_cancelled").
response.outputConversationItem[]Completed output items with content.
response.conversation_idstringConversation this response belongs to.
response.output_modalitiesstring[]"text", "audio", or both.
response.max_output_tokensinteger | "inf"Token limit for this response.
response.audioobjectAudio output config.
response.usageobject | nullToken usage statistics.
response.metadataobjectClient metadata from response.create, echoed unchanged. Omitted when not supplied.

response.output_item.added

An output item was added to the response.

FieldTypeDescription
typeresponse.output_item.addedEvent type.
event_idstringServer-generated event ID.
response_idstringThe response this item belongs to.
output_indexintegerIndex of the output item in the response.
itemConversationItemThe output item (initially empty content).

response.output_item.done

An output item finished.

FieldTypeDescription
typeresponse.output_item.doneEvent type.
event_idstringServer-generated event ID.
response_idstringThe response this item belongs to.
output_indexintegerIndex of the output item in the response.
itemConversationItemThe completed output item with content.

response.content_part.added

A content part was added to an output item.

FieldTypeDescription
typeresponse.content_part.addedEvent type.
event_idstringServer-generated event ID.
response_idstringResponse identifier.
item_idstringItem identifier.
output_indexintegerIndex of the output item.
content_indexintegerIndex of the content part.
partobjectThe content part. type: "audio" or "text". transcript: initially empty string.

response.content_part.done

A content part finished.

FieldTypeDescription
typeresponse.content_part.doneEvent type.
event_idstringServer-generated event ID.
response_idstringResponse identifier.
item_idstringItem identifier.
output_indexintegerIndex of the output item.
content_indexintegerIndex of the content part.
partobjectThe completed content part with final transcript.

response.output_text.delta

Streaming text chunk from the model.

FieldTypeDescription
typeresponse.output_text.deltaEvent type.
event_idstringServer-generated event ID.
response_idstringResponse identifier.
item_idstringItem identifier.
output_indexintegerIndex of the output item.
content_indexintegerIndex of the content part.
deltastringText chunk.

response.output_text.done

Text output finished.

FieldTypeDescription
typeresponse.output_text.doneEvent type.
event_idstringServer-generated event ID.
response_idstringResponse identifier.
item_idstringItem identifier.
output_indexintegerIndex of the output item.
content_indexintegerIndex of the content part.
textstringComplete text output.

response.output_audio_transcript.delta

Streaming transcript for generated audio.

FieldTypeDescription
typeresponse.output_audio_transcript.deltaEvent type.
event_idstringServer-generated event ID.
response_idstringResponse identifier.
item_idstringItem identifier.
output_indexintegerIndex of the output item.
content_indexintegerIndex of the content part.
deltastringTranscript chunk.

response.output_audio_transcript.done

Final transcript for generated audio.

FieldTypeDescription
typeresponse.output_audio_transcript.doneEvent type.
event_idstringServer-generated event ID.
response_idstringResponse identifier.
item_idstringItem identifier.
output_indexintegerIndex of the output item.
content_indexintegerIndex of the content part.
transcriptstringComplete transcript.

response.output_audio.delta

Streaming alignment data for generated speech. Over WebRTC, audio travels on the RTP media track. This data-channel event carries only timestamp_info; the delta field is empty. The event is present only when providerData.tts.timestamp_type is set.

FieldTypeDescription
typeresponse.output_audio.deltaEvent type.
event_idstringServer-generated event ID.
response_idstringResponse identifier.
item_idstringItem identifier.
output_indexintegerIndex of the output item.
content_indexintegerIndex of the content part.
deltastringAlways empty over WebRTC (audio is on the media track).
timestamp_infoobjectTTS alignment data. Contains word_alignment or character_alignment. See TTS timestamps and alignment.

response.output_audio.done

Audio output for a content part finished.

FieldTypeDescription
typeresponse.output_audio.doneEvent type.
event_idstringServer-generated event ID.
response_idstringResponse identifier.
item_idstringItem identifier.
output_indexintegerIndex of the output item.
content_indexintegerIndex of the content part.

response.function_call_arguments.delta

Streaming tool call arguments.

FieldTypeDescription
typeresponse.function_call_arguments.deltaEvent type.
event_idstringServer-generated event ID.
response_idstringResponse identifier.
item_idstringItem identifier.
output_indexintegerIndex of the output item.
call_idstringTool-call identifier.
deltastringArguments chunk (JSON string fragment).

response.function_call_arguments.done

Tool call arguments finished.

FieldTypeDescription
typeresponse.function_call_arguments.doneEvent type.
event_idstringServer-generated event ID.
response_idstringResponse identifier.
item_idstringItem identifier.
output_indexintegerIndex of the output item.
call_idstringTool-call identifier.
namestringFunction name.
argumentsstringComplete tool call arguments (JSON string).

input_audio_buffer.speech_started

Voice activity detected — user started speaking.

FieldTypeDescription
typeinput_audio_buffer.speech_startedEvent type.
event_idstringServer-generated event ID.
audio_start_msintegerMillisecond offset in the audio stream where speech was detected.
item_idstringThe conversation item ID associated with this speech segment.

input_audio_buffer.speech_stopped

Voice activity ended — user stopped speaking.

FieldTypeDescription
typeinput_audio_buffer.speech_stoppedEvent type.
event_idstringServer-generated event ID.
audio_end_msintegerMillisecond offset in the audio stream where speech ended.
item_idstringThe conversation item ID associated with this speech segment.

input_audio_buffer.committed

Buffered audio was committed as a conversation item.

FieldTypeDescription
typeinput_audio_buffer.committedEvent type.
event_idstringServer-generated event ID.
previous_item_idstringThe ID of the preceding conversation item, or null.
item_idstringThe new conversation item ID for the committed audio.

input_audio_buffer.timeout_triggered

The input buffer reached its idle timeout. This event applies only to server VAD and is controlled by turn_detection.idle_timeout_ms.

FieldTypeDescription
typeinput_audio_buffer.timeout_triggeredEvent type.
event_idstringServer-generated event ID.
audio_start_msintegerAudio buffer start offset (ms) at the time the idle timeout fired.
audio_end_msintegerAudio buffer end offset (ms) at the time the idle timeout fired.
item_idstringConversation item ID associated with the idle audio buffer.

input_audio_buffer.turn_suggestion

The server VAD smart-turn detector predicts an end-of-turn boundary. Use this signal for low-latency UI cues or to pre-warm a response before the final speech_stopped commit. If the user resumes speaking, input_audio_buffer.turn_suggestion_revoked may follow.

FieldTypeDescription
typeinput_audio_buffer.turn_suggestionEvent type.
event_idstringServer-generated event ID.
item_idstringThe conversation item ID associated with this utterance.
utterance_indexintegerMonotonic index of the utterance within the session. Pairs with the matching turn_suggestion_revoked.
probabilitynumberSmart-turn model end-of-turn probability (0.0–1.0).
trailing_silence_msnumberTrailing silence at the time of inference, in milliseconds.
audio_duration_msnumberAudio duration of the utterance at the time of inference, in milliseconds.
inference_msnumberSmart-turn model inference latency, in milliseconds.

input_audio_buffer.turn_suggestion_revoked

This event is emitted when the user resumes speaking after a turn_suggestion. It pairs with the most recent turn_suggestion that has the same utterance_index.

FieldTypeDescription
typeinput_audio_buffer.turn_suggestion_revokedEvent type.
event_idstringServer-generated event ID.
item_idstringThe conversation item ID associated with this utterance.
utterance_indexintegerIndex of the utterance whose previous turn_suggestion is being revoked.

Other audio buffer events

Event typeDescription
input_audio_buffer.clearedInput audio buffer was cleared.
output_audio_buffer.startedServer started sending output audio.
output_audio_buffer.stoppedServer stopped sending output audio.
output_audio_buffer.clearedOutput audio buffer was cleared.

response.backchannel.audio.delta

A streaming PCM audio chunk for a low-latency back-channel interjection, such as "uh-huh" or "right", while the user is speaking. It is separate from the main response stream. Use backchannel_id to group chunks from the same interjection.

FieldTypeDescription
typeresponse.backchannel.audio.deltaEvent type.
event_idstringServer-generated event ID.
backchannel_idstringSynthetic ID grouping deltas + done for a single back-channel interjection. Use as the playback bucket key so chunks of one interjection don't collide with the active response item.
deltastringBase64-encoded audio chunk in the session's configured audio.output.format (PCM16, audio/pcmu, or audio/pcma).

response.backchannel.audio.done

All audio for a back-channel interjection has been streamed. No teardown is required. Queued audio plays until the queue is empty.

FieldTypeDescription
typeresponse.backchannel.audio.doneEvent type.
event_idstringServer-generated event ID.
backchannel_idstringIdentifies which back-channel interjection finished streaming.
phrasestringThe chosen back-channel utterance (e.g. "uh-huh"). Optional — omitted when the decider doesn't surface the phrase to clients.

response.backchannel.skipped

A back-channel evaluation chose not to emit an interjection. Use this event for client-side telemetry, or ignore it if you do not need that information.

FieldTypeDescription
typeresponse.backchannel.skippedEvent type.
event_idstringServer-generated event ID.
reasonstringShort machine-readable string describing why no back-channel was emitted on this evaluation tick (e.g. min_gap_not_elapsed, deadline_missed, no_phrase). Stable enough for telemetry.

rate_limits.updated

Reports current rate limit state.

FieldTypeDescription
typerate_limits.updatedEvent type.
event_idstringServer-generated event ID.

Schemas

Session object

The session object configures model behavior, audio settings, tools, and more. It appears in signaling requests, session.update, session.created, and session.updated events.

FieldTypeDescription
objectrealtime.sessionObject type identifier (read-only).
typerealtimeFixed value.
idstringServer-assigned session ID (read-only).
modelstringModel identifier.
instructionsstringSystem prompt for the model.
output_modalitiesstring[]Output types: "text", "audio", or both.
temperaturenumberThe sampling temperature used for response generation.
max_output_tokensinteger | "inf"Maximum tokens per response (1–4096 or "inf").
audioAudioConfigAudio input/output settings.
toolsTool[]Function tools available to the model.
tool_choicestring | ToolChoiceTarget"none", "auto", "required", or a specific tool target.
truncationstring | object"auto", "disabled", or a retention_ratio config.
tracingstring | object"auto" or a tracing config with workflow_name, group_id, metadata.
includestring[]Optional data to include, e.g. "item.input_audio_transcription.logprobs".
text_generation_configobjectFine-grained LLM generation parameters including reasoning (effort, maxTokens, exclude). See text_generation_config.
providerDataobjectInworld extensions: stt, tts, memory, backchannel, responsiveness, and auto_tool_response (default true; set false for client-controlled continuation). See API Extensions.
expires_atintegerUnix timestamp for session expiration (read-only).

AudioConfig

FieldTypeDescription
input.formatAudioFormatInput audio format.
input.noise_reductionobjectNoise reduction config. type: "near_field" or "far_field".
input.transcriptionobjectTranscription config. model: transcription model identifier (e.g., inworld/inworld-stt-1). language: optional language code. prompts: optional array of expected words or phrases. Singular prompt is not supported by Inworld STT.
input.turn_detectionTurnDetectionTurn detection config.
output.formatAudioFormatOutput audio format.
output.voicestringVoice preset for audio output (e.g., Dennis). See the List Voices API or the Voice library page in the Inworld Portal for the full list of supported voices.
output.modelstringThe TTS model used for audio output.
output.speednumberPlayback speed (0.25–1.5).

AudioFormat

FieldTypeDescription
typestringMIME type: "audio/pcm", "audio/pcmu", or "audio/pcma".
rateintegerSample rate in Hz. Currently 24000.

TurnDetection

Turn detection has two modes, selected by the type field.

Server VAD (type: "server_vad"):

FieldTypeDescription
typeserver_vadMode selector.
thresholdnumberVAD sensitivity (0–1).
prefix_padding_msintegerMilliseconds of audio to include before speech onset.
silence_duration_msintegerSilence duration (ms) before speech is considered ended.
create_responsebooleanAuto-trigger response.create after speech ends.
interrupt_responsebooleanAllow new speech to interrupt active responses.
idle_timeout_msintegerIdle timeout in milliseconds.

Semantic VAD (type: "semantic_vad"):

FieldTypeDescription
typesemantic_vadMode selector.
eagernessstring"low", "medium", or "high".
create_responsebooleanAuto-trigger response.create after speech ends.
interrupt_responsebooleanAllow new speech to interrupt active responses.

ConversationItem

FieldTypeRequiredDescription
objectrealtime.itemNoObject type identifier (read-only, present in server responses).
idstringNoItem ID.
typestringYes"message", "function_call", or "function_call_output".
statusstringNo"completed", "in_progress", or "incomplete" (read-only, present in server responses).
rolestringNo"system", "user", "assistant", or "tool".
contentContentPart[]NoArray of content parts.
call_idstringVariant-specificOptional for function_call; required and non-empty for client-created function_call_output.
namestringVariant-specificRequired and non-empty for client-created function_call; always present on server snapshots.
argumentsstringNoMay be omitted on input and then normalizes to ""; always present on server function_call snapshots.
outputstringNoMay be omitted on input and then normalizes to ""; always present on server function_call_output snapshots.

ContentPart

FieldTypeRequiredDescription
typestringYesUser text/audio input uses input_text/input_audio; assistant item snapshots use output_text/output_audio; streaming response.content_part.* events use text/audio. Legacy text/audio aliases are accepted for assistant item creation.
textstringNoText content.
audiostringNoBase64-encoded audio.
transcriptstringNoHuman-readable transcript accompanying audio.

ResponseConfig

Per-response overrides for session defaults.

FieldTypeDescription
conversationstring"auto" or a conversation ID.
output_modalitiesstring[]"text", "audio", or both.
instructionsstringOverride instructions for this response.
voicestringOverride voice for this response.
max_output_tokensinteger | "inf"Override max tokens.
tool_choicestring | ToolChoiceTargetOverride tool choice.
toolsTool[]Override available tools.
metadataobject (string → string)Client metadata echoed on response.created and response.done. Maximum 16 entries; keys up to 64 Unicode code points and values up to 512.

Tool

FieldTypeRequiredDescription
typefunctionYesTool type.
namestringYesFunction name.
descriptionstringNoWhat the function does.
parametersobjectNoJSON Schema for function parameters.

ToolChoiceTarget

Specifies tool choice behavior. The server always returns this as an object.

FieldTypeRequiredDescription
typestringYes"auto", "none", "required", "function", or "mcp".
namestringNoFunction name (when type is "function").
server_labelstringNoMCP server label (when type is "mcp").