> ## Documentation Index > Fetch the complete documentation index at: https://docs.inworld.ai/llms.txt > Use this file to discover all available pages before exploring further. # Realtime API (WebSocket) > Real-time, multimodal AI interactions over WebSocket. Enables low-latency speech-to-speech conversations with AI models, supporting both audio and text modalities. The API maintains a persistent WebSocket connection where clients can: - Create and configure sessions with custom instructions and voice settings - Stream audio input in real-time for natural voice conversations - Send text input as an alternative to audio - Receive streaming audio and text responses with low latency - Manage conversation flow with turn detection and response control **Key Features:** - **Low Latency**: Optimized for real-time interactions - **Multimodal**: Supports both audio and text input/output - **Voice Activity Detection**: Automatic speech detection with configurable thresholds - **Streaming Responses**: Receive response events as they're generated - **Session Management**: Maintain conversation context across multiple interactions **Default Rate Limits:** - **Max Concurrent Sessions**: 20 sessions per account - **Max Packets Per Second**: 1,000 packets per second, shared across all sessions in an account This API implements the Realtime interface. Refer to the [Realtime overview](/realtime/overview) for hands-on guides. ## AsyncAPI ````yaml api-reference/realtimeAPI/realtime-websocket.json realtimeStream id: realtimeStream title: Realtime stream description: Primary WebSocket channel used for realtime sessions. servers: - id: production protocol: wss host: api.inworld.ai bindings: [] variables: [] address: /api/v1/realtime/session parameters: [] bindings: [] operations: - &ref_11 id: clientEvents title: Client events description: Events clients send over the WebSocket connection. type: receive messages: - &ref_13 id: sessionUpdate payload: - name: session.update description: >- Update the session configuration. The server responds with a session.updated event. type: object properties: - name: const type: string description: session.update required: false - name: event_id type: string description: Optional client-generated event ID. required: false - name: session type: object description: >- The session object configures model behavior, audio settings, tools, and more. It appears in session.update, session.created, and session.updated events. required: true properties: - name: object type: string description: Object type identifier (read-only). required: false - name: type type: string description: Fixed value. required: false - name: id type: string description: Server-assigned session ID (read-only). required: false - name: model type: string description: Model identifier. required: false - name: instructions type: string description: System instructions for the model. required: false - name: output_modalities type: array description: 'Output types: "text", "audio", or both.' required: false - name: temperature type: number description: The sampling temperature used for response generation. required: false - name: oneOf type: oneOf description: Must be one of these types properties: - name: type type: string description: integer required: false - name: minimum type: number required: false - name: maximum type: number required: false - name: type type: string description: string required: false - name: const type: string description: inf required: false - name: description type: string description: Maximum tokens per response (1–4096 or "inf"). required: false - name: audio type: object required: false properties: - name: input type: object required: false properties: - name: format type: object required: false properties: - name: type type: string description: >- MIME type: "audio/pcm", "audio/pcmu", or "audio/pcma". enumValues: - audio/pcm - audio/pcmu - audio/pcma required: false - name: rate type: integer description: >- Sample rate in Hz. Required for audio/pcm (default 24000). Ignored for audio/pcmu and audio/pcma, which are fixed at 8000. required: false - name: noise_reduction type: object required: false properties: - name: type type: string description: Noise reduction mode. enumValues: - near_field - far_field required: false - name: transcription type: object required: false properties: - name: model type: string description: >- Transcription model identifier (e.g., `assemblyai/universal-streaming-multilingual`). required: false - name: language type: string description: Optional language code. required: false - name: prompt type: string description: Optional transcription prompt. required: false - name: oneOf type: oneOf description: Must be one of these types properties: - name: const type: string description: server_vad required: false - name: threshold type: number description: VAD sensitivity (0–1). required: false - name: prefix_padding_ms type: integer description: >- Milliseconds of audio to include before speech onset. required: false - name: silence_duration_ms type: integer description: >- Silence duration (ms) before speech is considered ended. required: false - name: create_response type: boolean description: >- Auto-trigger response.create after speech ends. required: false - name: interrupt_response type: boolean description: >- Allow new speech to interrupt active responses. required: false - name: idle_timeout_ms type: integer description: Idle timeout in milliseconds. required: false - name: const type: string description: semantic_vad required: false - name: eagerness type: string description: Eagerness level. enumValues: - low - medium - high - auto required: false - name: create_response type: boolean description: >- Auto-trigger response.create after speech ends. required: false - name: interrupt_response type: boolean description: >- Allow new speech to interrupt active responses. required: false - name: output type: object required: false properties: - name: format type: object required: false properties: - name: type type: string description: >- MIME type: "audio/pcm", "audio/pcmu", or "audio/pcma". enumValues: - audio/pcm - audio/pcmu - audio/pcma required: false - name: rate type: integer description: >- Sample rate in Hz. Required for audio/pcm (default 24000). Ignored for audio/pcmu and audio/pcma, which are fixed at 8000. required: false - name: voice type: string description: >- Voice preset for audio output (e.g., `Dennis`). See the [List Voices](https://docs.inworld.ai/api-reference/voiceAPI/voiceservice/list-voices) API or the [Voice library](https://platform.inworld.ai/voice-library) page in the Inworld Portal for the full list of supported voices. required: false - name: model type: string description: The TTS model used for audio output. required: false - name: speed type: number description: Playback speed (0.25–1.5). required: false - name: tools type: array required: false - name: oneOf type: oneOf description: Must be one of these types properties: - name: type type: string description: string required: false - name: enum type: array required: false - name: type type: string description: Tool choice type. enumValues: - auto - none - required - function - mcp required: true - name: name type: string description: Function name (when type is "function"). required: false - name: server_label type: string description: MCP server label (when type is "mcp"). required: false - name: oneOf type: oneOf description: Must be one of these types properties: - name: type type: string description: string required: false - name: enum type: array required: false - name: const type: string description: retention_ratio required: false - name: retention_ratio type: number required: true - name: token_limits type: object required: false properties: - name: post_instructions type: integer required: false - name: oneOf type: oneOf description: Must be one of these types properties: - name: type type: string description: string required: false - name: const type: string description: auto required: false - name: workflow_name type: string required: false - name: group_id type: string required: false - name: metadata type: object required: false - name: include type: array description: Optional data to include. required: false - name: prompt type: &ref_1 - string - 'null' description: Prompt template reference, or null. required: false - name: expires_at type: integer description: Unix timestamp for session expiration (read-only). required: false headers: [] jsonPayloadSchema: type: object properties: type: const: session.update x-parser-schema-id: event_id: type: string description: Optional client-generated event ID. x-parser-schema-id: session: &ref_4 type: object description: >- The session object configures model behavior, audio settings, tools, and more. It appears in session.update, session.created, and session.updated events. properties: object: type: string const: realtime.session description: Object type identifier (read-only). x-parser-schema-id: type: type: string const: realtime description: Fixed value. x-parser-schema-id: id: type: string description: Server-assigned session ID (read-only). x-parser-schema-id: model: type: string description: Model identifier. x-parser-schema-id: instructions: type: string description: System instructions for the model. x-parser-schema-id: output_modalities: type: array items: type: string enum: - text - audio x-parser-schema-id: description: 'Output types: "text", "audio", or both.' x-parser-schema-id: temperature: type: number description: The sampling temperature used for response generation. x-parser-schema-id: max_output_tokens: oneOf: - type: integer minimum: 1 maximum: 4096 x-parser-schema-id: - type: string const: inf x-parser-schema-id: description: Maximum tokens per response (1–4096 or "inf"). x-parser-schema-id: audio: type: object properties: input: type: object properties: format: &ref_0 type: object properties: type: type: string enum: - audio/pcm - audio/pcmu - audio/pcma description: >- MIME type: "audio/pcm", "audio/pcmu", or "audio/pcma". x-parser-schema-id: rate: type: integer description: >- Sample rate in Hz. Required for audio/pcm (default 24000). Ignored for audio/pcmu and audio/pcma, which are fixed at 8000. x-parser-schema-id: x-parser-schema-id: AudioFormat noise_reduction: type: object properties: type: type: string enum: - near_field - far_field description: Noise reduction mode. x-parser-schema-id: required: - type x-parser-schema-id: NoiseReduction transcription: type: object properties: model: type: string description: >- Transcription model identifier (e.g., `assemblyai/universal-streaming-multilingual`). x-parser-schema-id: language: type: string description: Optional language code. x-parser-schema-id: prompt: type: string description: Optional transcription prompt. x-parser-schema-id: x-parser-schema-id: InputAudioTranscription turn_detection: oneOf: - type: object properties: type: const: server_vad x-parser-schema-id: threshold: type: number minimum: 0 maximum: 1 description: VAD sensitivity (0–1). x-parser-schema-id: prefix_padding_ms: type: integer description: >- Milliseconds of audio to include before speech onset. x-parser-schema-id: silence_duration_ms: type: integer description: >- Silence duration (ms) before speech is considered ended. x-parser-schema-id: create_response: type: boolean description: >- Auto-trigger response.create after speech ends. x-parser-schema-id: interrupt_response: type: boolean description: >- Allow new speech to interrupt active responses. x-parser-schema-id: idle_timeout_ms: type: integer description: Idle timeout in milliseconds. x-parser-schema-id: required: - type x-parser-schema-id: - type: object properties: type: const: semantic_vad x-parser-schema-id: eagerness: type: string enum: - low - medium - high - auto description: Eagerness level. x-parser-schema-id: create_response: type: boolean description: >- Auto-trigger response.create after speech ends. x-parser-schema-id: interrupt_response: type: boolean description: >- Allow new speech to interrupt active responses. x-parser-schema-id: required: - type x-parser-schema-id: x-parser-schema-id: TurnDetection x-parser-schema-id: output: type: object properties: format: *ref_0 voice: type: string description: >- Voice preset for audio output (e.g., `Dennis`). See the [List Voices](https://docs.inworld.ai/api-reference/voiceAPI/voiceservice/list-voices) API or the [Voice library](https://platform.inworld.ai/voice-library) page in the Inworld Portal for the full list of supported voices. x-parser-schema-id: model: type: string description: The TTS model used for audio output. x-parser-schema-id: speed: type: number minimum: 0.25 maximum: 1.5 description: Playback speed (0.25–1.5). x-parser-schema-id: x-parser-schema-id: x-parser-schema-id: AudioConfig tools: type: array items: &ref_3 type: object properties: type: type: string const: function description: Tool type. x-parser-schema-id: name: type: string description: Function name. x-parser-schema-id: description: type: string description: What the function does. x-parser-schema-id: parameters: type: object additionalProperties: true description: JSON Schema for function parameters. x-parser-schema-id: required: - type - name x-parser-schema-id: Tool x-parser-schema-id: tool_choice: oneOf: - type: string enum: - none - auto - required x-parser-schema-id: - &ref_2 type: object description: Specifies tool choice behavior. properties: type: type: string enum: - auto - none - required - function - mcp description: Tool choice type. x-parser-schema-id: name: type: string description: Function name (when type is "function"). x-parser-schema-id: server_label: type: string description: MCP server label (when type is "mcp"). x-parser-schema-id: required: - type x-parser-schema-id: ToolChoiceTarget x-parser-schema-id: truncation: oneOf: - type: string enum: - auto - disabled x-parser-schema-id: - type: object properties: type: const: retention_ratio x-parser-schema-id: retention_ratio: type: number minimum: 0 maximum: 1 x-parser-schema-id: token_limits: type: object properties: post_instructions: type: integer x-parser-schema-id: x-parser-schema-id: required: - type - retention_ratio x-parser-schema-id: RetentionRatioTruncation x-parser-schema-id: tracing: oneOf: - type: string const: auto x-parser-schema-id: - type: object properties: workflow_name: type: string x-parser-schema-id: group_id: type: string x-parser-schema-id: metadata: type: object additionalProperties: true x-parser-schema-id: x-parser-schema-id: TracingConfig x-parser-schema-id: include: type: array items: type: string enum: - item.input_audio_transcription.logprobs x-parser-schema-id: description: Optional data to include. x-parser-schema-id: prompt: type: *ref_1 description: Prompt template reference, or null. x-parser-schema-id: expires_at: type: integer description: Unix timestamp for session expiration (read-only). x-parser-schema-id: x-parser-schema-id: Session required: - type - session examples: - type: session.update session: instructions: You are a friendly voice assistant. audio: input: transcription: model: assemblyai/universal-streaming-multilingual turn_detection: type: semantic_vad eagerness: medium create_response: true interrupt_response: true output: voice: Dennis speed: 1 x-parser-schema-id: SessionUpdateEvent title: session.update description: >- Update the session configuration. The server responds with a session.updated event. example: |- { "type": "session.update", "session": { "instructions": "You are a friendly voice assistant.", "audio": { "input": { "transcription": { "model": "assemblyai/universal-streaming-multilingual" }, "turn_detection": { "type": "semantic_vad", "eagerness": "medium", "create_response": true, "interrupt_response": true } }, "output": { "voice": "Dennis", "speed": 1 } } } } bindings: [] extensions: - id: x-parser-unique-object-id value: sessionUpdate - &ref_14 id: conversationItemCreate payload: - name: conversation.item.create description: Add a conversation item (message, function call result, etc.). type: object properties: - name: const type: string description: conversation.item.create required: false - name: event_id type: string description: Optional client-generated event ID. required: false - name: previous_item_id type: string description: Insert after this item ID. required: false - name: item type: object required: true properties: - name: object type: string description: >- Object type identifier (read-only, present in server responses). required: false - name: id type: string description: Item ID. required: false - name: type type: string description: Item type (e.g., "message", "function_call_result"). required: false - name: status type: string description: Item status (read-only, present in server responses). enumValues: - completed - in_progress required: false - name: role type: string enumValues: - system - user - assistant - tool required: false - name: content type: array required: false headers: [] jsonPayloadSchema: type: object properties: type: const: conversation.item.create x-parser-schema-id: event_id: type: string description: Optional client-generated event ID. x-parser-schema-id: previous_item_id: type: string description: Insert after this item ID. x-parser-schema-id: item: &ref_6 type: object properties: object: type: string const: realtime.item description: >- Object type identifier (read-only, present in server responses). x-parser-schema-id: id: type: string description: Item ID. x-parser-schema-id: type: type: string description: Item type (e.g., "message", "function_call_result"). x-parser-schema-id: status: type: string enum: - completed - in_progress description: Item status (read-only, present in server responses). x-parser-schema-id: role: type: string enum: - system - user - assistant - tool x-parser-schema-id: content: type: array items: type: object properties: type: type: string description: >- Content type (e.g., "input_text", "input_audio", "text", "audio"). x-parser-schema-id: text: type: string description: Text content. x-parser-schema-id: audio: type: string description: Base64-encoded audio. x-parser-schema-id: transcript: type: string description: Human-readable transcript accompanying audio. x-parser-schema-id: required: - type x-parser-schema-id: ContentPart x-parser-schema-id: required: - type x-parser-schema-id: ConversationItem required: - type - item examples: - type: conversation.item.create item: type: message role: user content: - type: input_text text: Hello, how are you? x-parser-schema-id: ConversationItemCreateEvent title: conversation.item.create description: Add a conversation item (message, function call result, etc.). example: |- { "type": "conversation.item.create", "item": { "type": "message", "role": "user", "content": [ { "type": "input_text", "text": "Hello, how are you?" } ] } } bindings: [] extensions: - id: x-parser-unique-object-id value: conversationItemCreate - &ref_15 id: conversationItemTruncate payload: - name: conversation.item.truncate description: Truncate an assistant message's audio. type: object properties: - name: const type: string description: conversation.item.truncate required: false - name: event_id type: string description: Optional client-generated event ID. required: false - name: item_id type: string description: The ID of the assistant message item to truncate. required: true - name: content_index type: integer description: Index of the content part to truncate. required: true - name: audio_end_ms type: integer description: Millisecond offset to truncate the audio at. required: true headers: [] jsonPayloadSchema: type: object properties: type: const: conversation.item.truncate x-parser-schema-id: event_id: type: string description: Optional client-generated event ID. x-parser-schema-id: item_id: type: string description: The ID of the assistant message item to truncate. x-parser-schema-id: content_index: type: integer description: Index of the content part to truncate. x-parser-schema-id: audio_end_ms: type: integer description: Millisecond offset to truncate the audio at. x-parser-schema-id: required: - type - item_id - content_index - audio_end_ms x-parser-schema-id: ConversationItemTruncateEvent title: conversation.item.truncate description: Truncate an assistant message's audio. example: |- { "const": "", "event_id": "", "item_id": "", "content_index": 123, "audio_end_ms": 123 } bindings: [] extensions: - id: x-parser-unique-object-id value: conversationItemTruncate - &ref_16 id: conversationItemDelete payload: - name: conversation.item.delete description: Delete a conversation item by ID. type: object properties: - name: const type: string description: conversation.item.delete required: false - name: event_id type: string description: Optional client-generated event ID. required: false - name: item_id type: string description: The ID of the conversation item to delete. required: true headers: [] jsonPayloadSchema: type: object properties: type: const: conversation.item.delete x-parser-schema-id: event_id: type: string description: Optional client-generated event ID. x-parser-schema-id: item_id: type: string description: The ID of the conversation item to delete. x-parser-schema-id: required: - type - item_id x-parser-schema-id: ConversationItemDeleteEvent title: conversation.item.delete description: Delete a conversation item by ID. example: |- { "const": "", "event_id": "", "item_id": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: conversationItemDelete - &ref_17 id: conversationItemRetrieve payload: - name: conversation.item.retrieve description: Retrieve a conversation item by ID. type: object properties: - name: const type: string description: conversation.item.retrieve required: false - name: event_id type: string description: Optional client-generated event ID. required: false - name: item_id type: string description: The ID of the conversation item to retrieve. required: true headers: [] jsonPayloadSchema: type: object properties: type: const: conversation.item.retrieve x-parser-schema-id: event_id: type: string description: Optional client-generated event ID. x-parser-schema-id: item_id: type: string description: The ID of the conversation item to retrieve. x-parser-schema-id: required: - type - item_id x-parser-schema-id: ConversationItemRetrieveEvent title: conversation.item.retrieve description: Retrieve a conversation item by ID. example: |- { "const": "", "event_id": "", "item_id": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: conversationItemRetrieve - &ref_18 id: responseCreate payload: - name: response.create description: Trigger a model response. The server streams back response events. type: object properties: - name: const type: string description: response.create required: false - name: event_id type: string description: Optional client-generated event ID. required: false - name: response type: object description: Per-response overrides for session defaults. required: false properties: - name: oneOf type: oneOf description: Must be one of these types properties: - name: type type: string description: string required: false - name: const type: string description: auto required: false - name: type type: string description: string required: false - name: description type: string description: '"auto" or a conversation ID.' required: false - name: output_modalities type: array description: '"text", "audio", or both.' required: false - name: instructions type: string description: Override instructions for this response. required: false - name: voice type: string description: Override voice for this response. required: false - name: oneOf type: oneOf description: Must be one of these types properties: - name: type type: string description: integer required: false - name: minimum type: number required: false - name: maximum type: number required: false - name: type type: string description: string required: false - name: const type: string description: inf required: false - name: description type: string description: Override max tokens. required: false - name: oneOf type: oneOf description: Must be one of these types properties: - name: type type: string description: string required: false - name: enum type: array required: false - name: type type: string description: Tool choice type. enumValues: - auto - none - required - function - mcp required: true - name: name type: string description: Function name (when type is "function"). required: false - name: server_label type: string description: MCP server label (when type is "mcp"). required: false - name: description type: string description: Override tool choice. required: false - name: tools type: array description: Override available tools. required: false headers: [] jsonPayloadSchema: type: object properties: type: const: response.create x-parser-schema-id: event_id: type: string description: Optional client-generated event ID. x-parser-schema-id: response: type: object description: Per-response overrides for session defaults. properties: conversation: oneOf: - type: string const: auto x-parser-schema-id: - type: string x-parser-schema-id: description: '"auto" or a conversation ID.' x-parser-schema-id: output_modalities: type: array items: type: string enum: - text - audio x-parser-schema-id: description: '"text", "audio", or both.' x-parser-schema-id: instructions: type: string description: Override instructions for this response. x-parser-schema-id: voice: type: string description: Override voice for this response. x-parser-schema-id: max_output_tokens: oneOf: - type: integer minimum: 1 maximum: 4096 x-parser-schema-id: - type: string const: inf x-parser-schema-id: description: Override max tokens. x-parser-schema-id: tool_choice: oneOf: - type: string enum: - none - auto - required x-parser-schema-id: - *ref_2 description: Override tool choice. x-parser-schema-id: tools: type: array items: *ref_3 description: Override available tools. x-parser-schema-id: x-parser-schema-id: ResponseConfig required: - type examples: - type: response.create response: output_modalities: - audio - text instructions: Respond in a cheerful tone. x-parser-schema-id: ResponseCreateEvent title: response.create description: Trigger a model response. The server streams back response events. example: |- { "type": "response.create", "response": { "output_modalities": [ "audio", "text" ], "instructions": "Respond in a cheerful tone." } } bindings: [] extensions: - id: x-parser-unique-object-id value: responseCreate - &ref_19 id: responseCancel payload: - name: response.cancel description: Cancel an in-progress response. type: object properties: - name: const type: string description: response.cancel required: false - name: event_id type: string description: Optional client-generated event ID. required: false - name: response_id type: string description: >- Cancel a specific response by ID. If omitted, cancels the active response. required: false headers: [] jsonPayloadSchema: type: object properties: type: const: response.cancel x-parser-schema-id: event_id: type: string description: Optional client-generated event ID. x-parser-schema-id: response_id: type: string description: >- Cancel a specific response by ID. If omitted, cancels the active response. x-parser-schema-id: required: - type x-parser-schema-id: ResponseCancelEvent title: response.cancel description: Cancel an in-progress response. example: |- { "const": "", "event_id": "", "response_id": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: responseCancel - &ref_20 id: inputAudioAppend payload: - name: input_audio_buffer.append description: Append audio bytes to the input buffer. type: object properties: - name: const type: string description: input_audio_buffer.append required: false - name: event_id type: string description: Optional client-generated event ID. required: false - name: audio type: string description: >- Base64-encoded audio chunk (~100–200ms) matching the configured input format. required: true headers: [] jsonPayloadSchema: type: object properties: type: const: input_audio_buffer.append x-parser-schema-id: event_id: type: string description: Optional client-generated event ID. x-parser-schema-id: audio: type: string description: >- Base64-encoded audio chunk (~100–200ms) matching the configured input format. x-parser-schema-id: required: - type - audio x-parser-schema-id: InputAudioAppendEvent title: input_audio_buffer.append description: Append audio bytes to the input buffer. example: |- { "const": "", "event_id": "", "audio": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: inputAudioAppend - &ref_21 id: inputAudioCommit payload: - name: input_audio_buffer.commit description: Commit the buffered audio as a user message. type: object properties: - name: const type: string description: input_audio_buffer.commit required: false - name: event_id type: string required: false headers: [] jsonPayloadSchema: type: object properties: type: const: input_audio_buffer.commit x-parser-schema-id: event_id: type: string x-parser-schema-id: required: - type x-parser-schema-id: TypeOnlyInputAudioCommitEvent title: input_audio_buffer.commit description: Commit the buffered audio as a user message. example: |- { "const": "", "event_id": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: inputAudioCommit - &ref_22 id: inputAudioClear payload: - name: input_audio_buffer.clear description: Discard all audio in the input buffer. type: object properties: - name: const type: string description: input_audio_buffer.clear required: false - name: event_id type: string required: false headers: [] jsonPayloadSchema: type: object properties: type: const: input_audio_buffer.clear x-parser-schema-id: event_id: type: string x-parser-schema-id: required: - type x-parser-schema-id: TypeOnlyInputAudioClearEvent title: input_audio_buffer.clear description: Discard all audio in the input buffer. example: |- { "const": "", "event_id": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: inputAudioClear - &ref_23 id: outputAudioClear payload: - name: output_audio_buffer.clear description: Clear the server's output audio buffer, stopping playback. type: object properties: - name: const type: string description: output_audio_buffer.clear required: false - name: event_id type: string required: false headers: [] jsonPayloadSchema: type: object properties: type: const: output_audio_buffer.clear x-parser-schema-id: event_id: type: string x-parser-schema-id: required: - type x-parser-schema-id: TypeOnlyOutputAudioClearEvent title: output_audio_buffer.clear description: Clear the server's output audio buffer, stopping playback. example: |- { "const": "", "event_id": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: outputAudioClear bindings: [] extensions: &ref_10 - id: x-parser-unique-object-id value: realtimeStream - &ref_12 id: serverEvents title: Server events description: Events server emits over the WebSocket connection. type: send messages: - &ref_24 id: sessionCreated payload: - name: session.created description: >- Not currently supported. The session starts immediately with default configuration. Send a session.update to configure the session. type: object properties: - name: const type: string description: session.created required: false - name: event_id type: string description: Server-generated event ID. required: true - name: session type: object description: >- The session object configures model behavior, audio settings, tools, and more. It appears in session.update, session.created, and session.updated events. required: true properties: - name: object type: string description: Object type identifier (read-only). required: false - name: type type: string description: Fixed value. required: false - name: id type: string description: Server-assigned session ID (read-only). required: false - name: model type: string description: Model identifier. required: false - name: instructions type: string description: System instructions for the model. required: false - name: output_modalities type: array description: 'Output types: "text", "audio", or both.' required: false - name: temperature type: number description: The sampling temperature used for response generation. required: false - name: oneOf type: oneOf description: Must be one of these types properties: - name: type type: string description: integer required: false - name: minimum type: number required: false - name: maximum type: number required: false - name: type type: string description: string required: false - name: const type: string description: inf required: false - name: description type: string description: Maximum tokens per response (1–4096 or "inf"). required: false - name: audio type: object required: false properties: - name: input type: object required: false properties: - name: format type: object required: false properties: - name: type type: string description: >- MIME type: "audio/pcm", "audio/pcmu", or "audio/pcma". enumValues: - audio/pcm - audio/pcmu - audio/pcma required: false - name: rate type: integer description: >- Sample rate in Hz. Required for audio/pcm (default 24000). Ignored for audio/pcmu and audio/pcma, which are fixed at 8000. required: false - name: noise_reduction type: object required: false properties: - name: type type: string description: Noise reduction mode. enumValues: - near_field - far_field required: false - name: transcription type: object required: false properties: - name: model type: string description: >- Transcription model identifier (e.g., `assemblyai/universal-streaming-multilingual`). required: false - name: language type: string description: Optional language code. required: false - name: prompt type: string description: Optional transcription prompt. required: false - name: oneOf type: oneOf description: Must be one of these types properties: - name: const type: string description: server_vad required: false - name: threshold type: number description: VAD sensitivity (0–1). required: false - name: prefix_padding_ms type: integer description: >- Milliseconds of audio to include before speech onset. required: false - name: silence_duration_ms type: integer description: >- Silence duration (ms) before speech is considered ended. required: false - name: create_response type: boolean description: >- Auto-trigger response.create after speech ends. required: false - name: interrupt_response type: boolean description: >- Allow new speech to interrupt active responses. required: false - name: idle_timeout_ms type: integer description: Idle timeout in milliseconds. required: false - name: const type: string description: semantic_vad required: false - name: eagerness type: string description: Eagerness level. enumValues: - low - medium - high - auto required: false - name: create_response type: boolean description: >- Auto-trigger response.create after speech ends. required: false - name: interrupt_response type: boolean description: >- Allow new speech to interrupt active responses. required: false - name: output type: object required: false properties: - name: format type: object required: false properties: - name: type type: string description: >- MIME type: "audio/pcm", "audio/pcmu", or "audio/pcma". enumValues: - audio/pcm - audio/pcmu - audio/pcma required: false - name: rate type: integer description: >- Sample rate in Hz. Required for audio/pcm (default 24000). Ignored for audio/pcmu and audio/pcma, which are fixed at 8000. required: false - name: voice type: string description: >- Voice preset for audio output (e.g., `Dennis`). See the [List Voices](https://docs.inworld.ai/api-reference/voiceAPI/voiceservice/list-voices) API or the [Voice library](https://platform.inworld.ai/voice-library) page in the Inworld Portal for the full list of supported voices. required: false - name: model type: string description: The TTS model used for audio output. required: false - name: speed type: number description: Playback speed (0.25–1.5). required: false - name: tools type: array required: false - name: oneOf type: oneOf description: Must be one of these types properties: - name: type type: string description: string required: false - name: enum type: array required: false - name: type type: string description: Tool choice type. enumValues: - auto - none - required - function - mcp required: true - name: name type: string description: Function name (when type is "function"). required: false - name: server_label type: string description: MCP server label (when type is "mcp"). required: false - name: oneOf type: oneOf description: Must be one of these types properties: - name: type type: string description: string required: false - name: enum type: array required: false - name: const type: string description: retention_ratio required: false - name: retention_ratio type: number required: true - name: token_limits type: object required: false properties: - name: post_instructions type: integer required: false - name: oneOf type: oneOf description: Must be one of these types properties: - name: type type: string description: string required: false - name: const type: string description: auto required: false - name: workflow_name type: string required: false - name: group_id type: string required: false - name: metadata type: object required: false - name: include type: array description: Optional data to include. required: false - name: prompt type: *ref_1 description: Prompt template reference, or null. required: false - name: expires_at type: integer description: Unix timestamp for session expiration (read-only). required: false headers: [] jsonPayloadSchema: type: object description: >- Not currently supported. The session starts immediately with default configuration. Send a session.update to configure the session. properties: type: const: session.created x-parser-schema-id: event_id: type: string description: Server-generated event ID. x-parser-schema-id: session: *ref_4 required: - type - event_id - session x-parser-schema-id: SessionCreatedEvent title: session.created description: >- Not currently supported. The session starts immediately with default configuration. Send a session.update to configure the session. example: No examples found bindings: [] extensions: - id: x-parser-unique-object-id value: sessionCreated - &ref_25 id: sessionUpdated payload: - name: session.updated description: Confirms a session.update was applied. type: object properties: - name: const type: string description: session.updated required: false - name: event_id type: string description: Server-generated event ID. required: true - name: session type: object description: >- The session object configures model behavior, audio settings, tools, and more. It appears in session.update, session.created, and session.updated events. required: true properties: - name: object type: string description: Object type identifier (read-only). required: false - name: type type: string description: Fixed value. required: false - name: id type: string description: Server-assigned session ID (read-only). required: false - name: model type: string description: Model identifier. required: false - name: instructions type: string description: System instructions for the model. required: false - name: output_modalities type: array description: 'Output types: "text", "audio", or both.' required: false - name: temperature type: number description: The sampling temperature used for response generation. required: false - name: oneOf type: oneOf description: Must be one of these types properties: - name: type type: string description: integer required: false - name: minimum type: number required: false - name: maximum type: number required: false - name: type type: string description: string required: false - name: const type: string description: inf required: false - name: description type: string description: Maximum tokens per response (1–4096 or "inf"). required: false - name: audio type: object required: false properties: - name: input type: object required: false properties: - name: format type: object required: false properties: - name: type type: string description: >- MIME type: "audio/pcm", "audio/pcmu", or "audio/pcma". enumValues: - audio/pcm - audio/pcmu - audio/pcma required: false - name: rate type: integer description: >- Sample rate in Hz. Required for audio/pcm (default 24000). Ignored for audio/pcmu and audio/pcma, which are fixed at 8000. required: false - name: noise_reduction type: object required: false properties: - name: type type: string description: Noise reduction mode. enumValues: - near_field - far_field required: false - name: transcription type: object required: false properties: - name: model type: string description: >- Transcription model identifier (e.g., `assemblyai/universal-streaming-multilingual`). required: false - name: language type: string description: Optional language code. required: false - name: prompt type: string description: Optional transcription prompt. required: false - name: oneOf type: oneOf description: Must be one of these types properties: - name: const type: string description: server_vad required: false - name: threshold type: number description: VAD sensitivity (0–1). required: false - name: prefix_padding_ms type: integer description: >- Milliseconds of audio to include before speech onset. required: false - name: silence_duration_ms type: integer description: >- Silence duration (ms) before speech is considered ended. required: false - name: create_response type: boolean description: >- Auto-trigger response.create after speech ends. required: false - name: interrupt_response type: boolean description: >- Allow new speech to interrupt active responses. required: false - name: idle_timeout_ms type: integer description: Idle timeout in milliseconds. required: false - name: const type: string description: semantic_vad required: false - name: eagerness type: string description: Eagerness level. enumValues: - low - medium - high - auto required: false - name: create_response type: boolean description: >- Auto-trigger response.create after speech ends. required: false - name: interrupt_response type: boolean description: >- Allow new speech to interrupt active responses. required: false - name: output type: object required: false properties: - name: format type: object required: false properties: - name: type type: string description: >- MIME type: "audio/pcm", "audio/pcmu", or "audio/pcma". enumValues: - audio/pcm - audio/pcmu - audio/pcma required: false - name: rate type: integer description: >- Sample rate in Hz. Required for audio/pcm (default 24000). Ignored for audio/pcmu and audio/pcma, which are fixed at 8000. required: false - name: voice type: string description: >- Voice preset for audio output (e.g., `Dennis`). See the [List Voices](https://docs.inworld.ai/api-reference/voiceAPI/voiceservice/list-voices) API or the [Voice library](https://platform.inworld.ai/voice-library) page in the Inworld Portal for the full list of supported voices. required: false - name: model type: string description: The TTS model used for audio output. required: false - name: speed type: number description: Playback speed (0.25–1.5). required: false - name: tools type: array required: false - name: oneOf type: oneOf description: Must be one of these types properties: - name: type type: string description: string required: false - name: enum type: array required: false - name: type type: string description: Tool choice type. enumValues: - auto - none - required - function - mcp required: true - name: name type: string description: Function name (when type is "function"). required: false - name: server_label type: string description: MCP server label (when type is "mcp"). required: false - name: oneOf type: oneOf description: Must be one of these types properties: - name: type type: string description: string required: false - name: enum type: array required: false - name: const type: string description: retention_ratio required: false - name: retention_ratio type: number required: true - name: token_limits type: object required: false properties: - name: post_instructions type: integer required: false - name: oneOf type: oneOf description: Must be one of these types properties: - name: type type: string description: string required: false - name: const type: string description: auto required: false - name: workflow_name type: string required: false - name: group_id type: string required: false - name: metadata type: object required: false - name: include type: array description: Optional data to include. required: false - name: prompt type: *ref_1 description: Prompt template reference, or null. required: false - name: expires_at type: integer description: Unix timestamp for session expiration (read-only). required: false headers: [] jsonPayloadSchema: type: object description: Confirms a session.update was applied. properties: type: const: session.updated x-parser-schema-id: event_id: type: string description: Server-generated event ID. x-parser-schema-id: session: *ref_4 required: - type - event_id - session x-parser-schema-id: SessionUpdatedEvent title: session.updated description: Confirms a session.update was applied. example: No examples found bindings: [] extensions: - id: x-parser-unique-object-id value: sessionUpdated - &ref_26 id: errorMessage payload: - name: error description: Indicates an error occurred. type: object properties: - name: const type: string description: error required: false - name: event_id type: string description: Server-generated event ID. required: true - name: error type: object required: true properties: - name: type type: string description: Error category. required: false - name: code type: string description: Error code. required: false - name: message type: string description: Human-readable error description. required: false - name: param type: string description: Related parameter, if applicable. required: false - name: event_id type: string description: The client event ID that caused the error, if applicable. required: false headers: [] jsonPayloadSchema: type: object properties: type: const: error x-parser-schema-id: event_id: type: string description: Server-generated event ID. x-parser-schema-id: error: type: object properties: type: type: string description: Error category. x-parser-schema-id: code: type: string description: Error code. x-parser-schema-id: message: type: string description: Human-readable error description. x-parser-schema-id: param: type: string description: Related parameter, if applicable. x-parser-schema-id: event_id: type: string description: The client event ID that caused the error, if applicable. x-parser-schema-id: required: - type - message x-parser-schema-id: required: - type - event_id - error examples: - event_id: 2c23cfd4-a4b5-4a96-83b8-a6a151f3989e type: error error: type: server_error code: null message: Failed to read content stream. param: null event_id: null x-parser-schema-id: ErrorEvent title: error description: Indicates an error occurred. example: |- { "event_id": "2c23cfd4-a4b5-4a96-83b8-a6a151f3989e", "type": "error", "error": { "type": "server_error", "code": null, "message": "Failed to read content stream.", "param": null, "event_id": null } } bindings: [] extensions: - id: x-parser-unique-object-id value: errorMessage - &ref_27 id: conversationItemAdded payload: - name: conversation.item.added description: A new item was added to the conversation. type: object properties: - name: const type: string description: conversation.item.added required: false - name: event_id type: string description: Server-generated event ID. required: true - name: previous_item_id type: &ref_5 - string - 'null' description: The ID of the preceding conversation item, or null. required: false - name: item type: object required: true properties: - name: object type: string description: >- Object type identifier (read-only, present in server responses). required: false - name: id type: string description: Item ID. required: false - name: type type: string description: Item type (e.g., "message", "function_call_result"). required: false - name: status type: string description: Item status (read-only, present in server responses). enumValues: - completed - in_progress required: false - name: role type: string enumValues: - system - user - assistant - tool required: false - name: content type: array required: false headers: [] jsonPayloadSchema: type: object properties: type: const: conversation.item.added x-parser-schema-id: event_id: type: string description: Server-generated event ID. x-parser-schema-id: previous_item_id: type: *ref_5 description: The ID of the preceding conversation item, or null. x-parser-schema-id: item: *ref_6 required: - type - event_id - item x-parser-schema-id: ConversationItemAddedEvent title: conversation.item.added description: A new item was added to the conversation. example: No examples found bindings: [] extensions: - id: x-parser-unique-object-id value: conversationItemAdded - &ref_28 id: conversationItemDone payload: - name: conversation.item.done description: An item finished being populated. type: object properties: - name: const type: string description: conversation.item.done required: false - name: event_id type: string description: Server-generated event ID. required: true - name: previous_item_id type: &ref_7 - string - 'null' description: The ID of the preceding conversation item, or null. required: false - name: item type: object required: true properties: - name: object type: string description: >- Object type identifier (read-only, present in server responses). required: false - name: id type: string description: Item ID. required: false - name: type type: string description: Item type (e.g., "message", "function_call_result"). required: false - name: status type: string description: Item status (read-only, present in server responses). enumValues: - completed - in_progress required: false - name: role type: string enumValues: - system - user - assistant - tool required: false - name: content type: array required: false headers: [] jsonPayloadSchema: type: object properties: type: const: conversation.item.done x-parser-schema-id: event_id: type: string description: Server-generated event ID. x-parser-schema-id: previous_item_id: type: *ref_7 description: The ID of the preceding conversation item, or null. x-parser-schema-id: item: *ref_6 required: - type - event_id - item x-parser-schema-id: ConversationItemDoneEvent title: conversation.item.done description: An item finished being populated. example: No examples found bindings: [] extensions: - id: x-parser-unique-object-id value: conversationItemDone - &ref_29 id: conversationItemDeleted payload: - name: conversation.item.deleted description: An item was deleted from the conversation. type: object properties: - name: const type: string description: conversation.item.deleted required: false - name: event_id type: string description: Server-generated event ID. required: true - name: item_id type: string description: The ID of the deleted item. required: true headers: [] jsonPayloadSchema: type: object properties: type: const: conversation.item.deleted x-parser-schema-id: event_id: type: string description: Server-generated event ID. x-parser-schema-id: item_id: type: string description: The ID of the deleted item. x-parser-schema-id: required: - type - event_id - item_id x-parser-schema-id: ConversationItemDeletedEvent title: conversation.item.deleted description: An item was deleted from the conversation. example: |- { "const": "", "event_id": "", "item_id": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: conversationItemDeleted - &ref_30 id: conversationItemRetrieved payload: - name: conversation.item.retrieved description: Response to conversation.item.retrieve. type: object properties: - name: const type: string description: conversation.item.retrieved required: false - name: event_id type: string required: true headers: [] jsonPayloadSchema: type: object properties: type: const: conversation.item.retrieved x-parser-schema-id: event_id: type: string x-parser-schema-id: required: - type - event_id x-parser-schema-id: TypeOnlyConversationItemRetrievedEvent title: conversation.item.retrieved description: Response to conversation.item.retrieve. example: |- { "const": "", "event_id": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: conversationItemRetrieved - &ref_31 id: conversationItemTruncated payload: - name: conversation.item.truncated description: An assistant audio item was truncated. type: object properties: - name: const type: string description: conversation.item.truncated required: false - name: event_id type: string required: true headers: [] jsonPayloadSchema: type: object properties: type: const: conversation.item.truncated x-parser-schema-id: event_id: type: string x-parser-schema-id: required: - type - event_id x-parser-schema-id: TypeOnlyConversationItemTruncatedEvent title: conversation.item.truncated description: An assistant audio item was truncated. example: |- { "const": "", "event_id": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: conversationItemTruncated - &ref_32 id: conversationItemInputAudioTranscriptionDelta payload: - name: conversation.item.input_audio_transcription.delta description: Streaming partial transcription for user audio. type: object properties: - name: const type: string description: conversation.item.input_audio_transcription.delta required: false - name: event_id type: string description: Server-generated event ID. required: true - name: item_id type: string description: The conversation item being transcribed. required: true - name: content_index type: integer description: Index of the content part being transcribed. required: false - name: delta type: string description: Partial transcription text. required: true headers: [] jsonPayloadSchema: type: object properties: type: const: conversation.item.input_audio_transcription.delta x-parser-schema-id: event_id: type: string description: Server-generated event ID. x-parser-schema-id: item_id: type: string description: The conversation item being transcribed. x-parser-schema-id: content_index: type: integer description: Index of the content part being transcribed. x-parser-schema-id: delta: type: string description: Partial transcription text. x-parser-schema-id: required: - type - event_id - item_id - delta x-parser-schema-id: ConversationItemInputAudioTranscriptionDeltaEvent title: conversation.item.input_audio_transcription.delta description: Streaming partial transcription for user audio. example: |- { "const": "", "event_id": "", "item_id": "", "content_index": 123, "delta": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: conversationItemInputAudioTranscriptionDelta - &ref_33 id: conversationItemInputAudioTranscriptionCompleted payload: - name: conversation.item.input_audio_transcription.completed description: Final transcription for a user audio item. type: object properties: - name: const type: string description: conversation.item.input_audio_transcription.completed required: false - name: event_id type: string description: Server-generated event ID. required: true - name: item_id type: string description: The conversation item that was transcribed. required: true - name: content_index type: integer description: Index of the content part that was transcribed. required: false - name: transcript type: string description: Complete transcription text. required: true headers: [] jsonPayloadSchema: type: object properties: type: const: conversation.item.input_audio_transcription.completed x-parser-schema-id: event_id: type: string description: Server-generated event ID. x-parser-schema-id: item_id: type: string description: The conversation item that was transcribed. x-parser-schema-id: content_index: type: integer description: Index of the content part that was transcribed. x-parser-schema-id: transcript: type: string description: Complete transcription text. x-parser-schema-id: required: - type - event_id - item_id - transcript x-parser-schema-id: ConversationItemInputAudioTranscriptionCompletedEvent title: conversation.item.input_audio_transcription.completed description: Final transcription for a user audio item. example: |- { "const": "", "event_id": "", "item_id": "", "content_index": 123, "transcript": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: conversationItemInputAudioTranscriptionCompleted - &ref_34 id: responseCreated payload: - name: response.created description: >- A new response was created. Contains the full response object in its initial state. type: object properties: - name: const type: string description: response.created required: false - name: event_id type: string description: Server-generated event ID. required: true - name: response type: object description: >- The response object returned in response.created and response.done events. required: true properties: - name: id type: string description: Response identifier. required: false - name: object type: string description: Object type. required: false - name: status type: string description: Response status. enumValues: - in_progress - completed - cancelled - failed required: false - name: oneOf type: oneOf description: Must be one of these types properties: - name: type type: string enumValues: - completed - cancelled - failed required: false - name: reason type: string required: false - name: error type: object required: false properties: - name: type type: string required: false - name: code type: string required: false - name: type type: string description: 'null' required: false - name: description type: string description: Status details, if any. required: false - name: output type: array description: Output items. required: false - name: conversation_id type: string description: Conversation this response belongs to. required: false - name: output_modalities type: array description: '"text", "audio", or both.' required: false - name: oneOf type: oneOf description: Must be one of these types properties: - name: type type: string description: integer required: false - name: type type: string description: string required: false - name: const type: string description: inf required: false - name: description type: string description: Token limit for this response. required: false - name: audio type: object description: Audio output config echoed from session. required: false - name: instructions type: string description: >- Instructions applied to this response (echoed from session). required: false - name: usage type: object description: Token usage (populated in response.done). required: false properties: - name: total_tokens type: integer required: false - name: input_tokens type: integer required: false - name: output_tokens type: integer required: false - name: input_token_details type: object required: false properties: - name: cached_tokens type: integer required: false - name: text_tokens type: integer required: false - name: audio_tokens type: integer required: false - name: output_token_details type: object required: false properties: - name: text_tokens type: integer required: false - name: audio_tokens type: integer required: false - name: metadata type: object description: Response metadata. required: false headers: [] jsonPayloadSchema: type: object properties: type: const: response.created x-parser-schema-id: event_id: type: string description: Server-generated event ID. x-parser-schema-id: response: &ref_8 type: object description: >- The response object returned in response.created and response.done events. properties: id: type: string description: Response identifier. x-parser-schema-id: object: type: string const: realtime.response description: Object type. x-parser-schema-id: status: type: string enum: - in_progress - completed - cancelled - failed description: Response status. x-parser-schema-id: status_details: oneOf: - type: object properties: type: type: string enum: - completed - cancelled - failed x-parser-schema-id: reason: type: string x-parser-schema-id: error: type: object properties: type: type: string x-parser-schema-id: code: type: string x-parser-schema-id: x-parser-schema-id: x-parser-schema-id: - type: 'null' x-parser-schema-id: description: Status details, if any. x-parser-schema-id: output: type: array items: *ref_6 description: Output items. x-parser-schema-id: conversation_id: type: string description: Conversation this response belongs to. x-parser-schema-id: output_modalities: type: array items: type: string x-parser-schema-id: description: '"text", "audio", or both.' x-parser-schema-id: max_output_tokens: oneOf: - type: integer x-parser-schema-id: - type: string const: inf x-parser-schema-id: description: Token limit for this response. x-parser-schema-id: audio: type: object description: Audio output config echoed from session. x-parser-schema-id: instructions: type: string description: Instructions applied to this response (echoed from session). x-parser-schema-id: usage: type: object nullable: true description: Token usage (populated in response.done). properties: total_tokens: type: integer x-parser-schema-id: input_tokens: type: integer x-parser-schema-id: output_tokens: type: integer x-parser-schema-id: input_token_details: type: object properties: cached_tokens: type: integer x-parser-schema-id: text_tokens: type: integer x-parser-schema-id: audio_tokens: type: integer x-parser-schema-id: x-parser-schema-id: output_token_details: type: object properties: text_tokens: type: integer x-parser-schema-id: audio_tokens: type: integer x-parser-schema-id: x-parser-schema-id: x-parser-schema-id: metadata: type: object nullable: true description: Response metadata. x-parser-schema-id: x-parser-schema-id: ResponseObject required: - type - event_id - response x-parser-schema-id: ResponseCreatedEvent title: response.created description: >- A new response was created. Contains the full response object in its initial state. example: No examples found bindings: [] extensions: - id: x-parser-unique-object-id value: responseCreated - &ref_35 id: responseDone payload: - name: response.done description: >- The response finished. Contains the completed response object with final status and output. type: object properties: - name: const type: string description: response.done required: false - name: event_id type: string description: Server-generated event ID. required: true - name: response type: object description: >- The response object returned in response.created and response.done events. required: true properties: - name: id type: string description: Response identifier. required: false - name: object type: string description: Object type. required: false - name: status type: string description: Response status. enumValues: - in_progress - completed - cancelled - failed required: false - name: oneOf type: oneOf description: Must be one of these types properties: - name: type type: string enumValues: - completed - cancelled - failed required: false - name: reason type: string required: false - name: error type: object required: false properties: - name: type type: string required: false - name: code type: string required: false - name: type type: string description: 'null' required: false - name: description type: string description: Status details, if any. required: false - name: output type: array description: Output items. required: false - name: conversation_id type: string description: Conversation this response belongs to. required: false - name: output_modalities type: array description: '"text", "audio", or both.' required: false - name: oneOf type: oneOf description: Must be one of these types properties: - name: type type: string description: integer required: false - name: type type: string description: string required: false - name: const type: string description: inf required: false - name: description type: string description: Token limit for this response. required: false - name: audio type: object description: Audio output config echoed from session. required: false - name: instructions type: string description: >- Instructions applied to this response (echoed from session). required: false - name: usage type: object description: Token usage (populated in response.done). required: false properties: - name: total_tokens type: integer required: false - name: input_tokens type: integer required: false - name: output_tokens type: integer required: false - name: input_token_details type: object required: false properties: - name: cached_tokens type: integer required: false - name: text_tokens type: integer required: false - name: audio_tokens type: integer required: false - name: output_token_details type: object required: false properties: - name: text_tokens type: integer required: false - name: audio_tokens type: integer required: false - name: metadata type: object description: Response metadata. required: false headers: [] jsonPayloadSchema: type: object properties: type: const: response.done x-parser-schema-id: event_id: type: string description: Server-generated event ID. x-parser-schema-id: response: *ref_8 required: - type - event_id - response x-parser-schema-id: ResponseDoneEvent title: response.done description: >- The response finished. Contains the completed response object with final status and output. example: No examples found bindings: [] extensions: - id: x-parser-unique-object-id value: responseDone - &ref_36 id: responseOutputItemAdded payload: - name: response.output_item.added description: An output item was added to the response. type: object properties: - name: const type: string description: response.output_item.added required: false - name: event_id type: string description: Server-generated event ID. required: true - name: response_id type: string description: The response this item belongs to. required: true - name: output_index type: integer description: Index of the output item in the response. required: false - name: item type: object required: true properties: - name: object type: string description: >- Object type identifier (read-only, present in server responses). required: false - name: id type: string description: Item ID. required: false - name: type type: string description: Item type (e.g., "message", "function_call_result"). required: false - name: status type: string description: Item status (read-only, present in server responses). enumValues: - completed - in_progress required: false - name: role type: string enumValues: - system - user - assistant - tool required: false - name: content type: array required: false headers: [] jsonPayloadSchema: type: object properties: type: const: response.output_item.added x-parser-schema-id: event_id: type: string description: Server-generated event ID. x-parser-schema-id: response_id: type: string description: The response this item belongs to. x-parser-schema-id: output_index: type: integer description: Index of the output item in the response. x-parser-schema-id: item: *ref_6 required: - type - event_id - response_id - item x-parser-schema-id: ResponseOutputItemAddedEvent title: response.output_item.added description: An output item was added to the response. example: No examples found bindings: [] extensions: - id: x-parser-unique-object-id value: responseOutputItemAdded - &ref_37 id: responseOutputItemDone payload: - name: response.output_item.done description: An output item finished. type: object properties: - name: const type: string description: response.output_item.done required: false - name: event_id type: string description: Server-generated event ID. required: true - name: response_id type: string description: The response this item belongs to. required: true - name: output_index type: integer description: Index of the output item in the response. required: false - name: item type: object required: true properties: - name: object type: string description: >- Object type identifier (read-only, present in server responses). required: false - name: id type: string description: Item ID. required: false - name: type type: string description: Item type (e.g., "message", "function_call_result"). required: false - name: status type: string description: Item status (read-only, present in server responses). enumValues: - completed - in_progress required: false - name: role type: string enumValues: - system - user - assistant - tool required: false - name: content type: array required: false headers: [] jsonPayloadSchema: type: object properties: type: const: response.output_item.done x-parser-schema-id: event_id: type: string description: Server-generated event ID. x-parser-schema-id: response_id: type: string description: The response this item belongs to. x-parser-schema-id: output_index: type: integer description: Index of the output item in the response. x-parser-schema-id: item: *ref_6 required: - type - event_id - response_id - item x-parser-schema-id: ResponseOutputItemDoneEvent title: response.output_item.done description: An output item finished. example: No examples found bindings: [] extensions: - id: x-parser-unique-object-id value: responseOutputItemDone - &ref_38 id: responseContentPartAdded payload: - name: response.content_part.added description: A content part was added to an output item. type: object properties: - name: const type: string description: response.content_part.added required: false - name: event_id type: string description: Server-generated event ID. required: true - name: response_id type: string description: Response identifier. required: true - name: item_id type: string description: Item identifier. required: false - name: output_index type: integer description: Index of the output item. required: false - name: content_index type: integer description: Index of the content part. required: false - name: part type: object description: >- The content part. type: "audio" or "text". transcript: initially empty string. required: true headers: [] jsonPayloadSchema: type: object properties: type: const: response.content_part.added x-parser-schema-id: event_id: type: string description: Server-generated event ID. x-parser-schema-id: response_id: type: string description: Response identifier. x-parser-schema-id: item_id: type: string description: Item identifier. x-parser-schema-id: output_index: type: integer description: Index of the output item. x-parser-schema-id: content_index: type: integer description: Index of the content part. x-parser-schema-id: part: type: object description: >- The content part. type: "audio" or "text". transcript: initially empty string. x-parser-schema-id: required: - type - event_id - response_id - part x-parser-schema-id: ResponseContentPartAddedEvent title: response.content_part.added description: A content part was added to an output item. example: No examples found bindings: [] extensions: - id: x-parser-unique-object-id value: responseContentPartAdded - &ref_39 id: responseContentPartDone payload: - name: response.content_part.done description: A content part finished. type: object properties: - name: const type: string description: response.content_part.done required: false - name: event_id type: string description: Server-generated event ID. required: true - name: response_id type: string description: Response identifier. required: true - name: item_id type: string description: Item identifier. required: false - name: output_index type: integer description: Index of the output item. required: false - name: content_index type: integer description: Index of the content part. required: false - name: part type: object description: The completed content part with final transcript. required: true headers: [] jsonPayloadSchema: type: object properties: type: const: response.content_part.done x-parser-schema-id: event_id: type: string description: Server-generated event ID. x-parser-schema-id: response_id: type: string description: Response identifier. x-parser-schema-id: item_id: type: string description: Item identifier. x-parser-schema-id: output_index: type: integer description: Index of the output item. x-parser-schema-id: content_index: type: integer description: Index of the content part. x-parser-schema-id: part: type: object description: The completed content part with final transcript. x-parser-schema-id: required: - type - event_id - response_id - part x-parser-schema-id: ResponseContentPartDoneEvent title: response.content_part.done description: A content part finished. example: No examples found bindings: [] extensions: - id: x-parser-unique-object-id value: responseContentPartDone - &ref_40 id: responseOutputTextDelta payload: - name: response.output_text.delta description: Streaming text chunk from the model. type: object properties: - name: const type: string description: response.output_text.delta required: false - name: event_id type: string description: Server-generated event ID. required: true - name: response_id type: string description: Response identifier. required: true - name: item_id type: string description: Item identifier. required: false - name: output_index type: integer description: Index of the output item. required: false - name: content_index type: integer description: Index of the content part. required: false - name: delta type: string description: Text chunk. required: true headers: [] jsonPayloadSchema: type: object properties: type: const: response.output_text.delta x-parser-schema-id: event_id: type: string description: Server-generated event ID. x-parser-schema-id: response_id: type: string description: Response identifier. x-parser-schema-id: item_id: type: string description: Item identifier. x-parser-schema-id: output_index: type: integer description: Index of the output item. x-parser-schema-id: content_index: type: integer description: Index of the content part. x-parser-schema-id: delta: type: string description: Text chunk. x-parser-schema-id: required: - type - event_id - response_id - delta x-parser-schema-id: ResponseOutputTextDeltaEvent title: response.output_text.delta description: Streaming text chunk from the model. example: |- { "const": "", "event_id": "", "response_id": "", "item_id": "", "output_index": 123, "content_index": 123, "delta": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: responseOutputTextDelta - &ref_41 id: responseOutputTextDone payload: - name: response.output_text.done description: Text output finished. type: object properties: - name: const type: string description: response.output_text.done required: false - name: event_id type: string description: Server-generated event ID. required: true - name: response_id type: string description: Response identifier. required: true - name: item_id type: string description: Item identifier. required: false - name: output_index type: integer description: Index of the output item. required: false - name: content_index type: integer description: Index of the content part. required: false - name: text type: string description: Complete text output. required: true headers: [] jsonPayloadSchema: type: object properties: type: const: response.output_text.done x-parser-schema-id: event_id: type: string description: Server-generated event ID. x-parser-schema-id: response_id: type: string description: Response identifier. x-parser-schema-id: item_id: type: string description: Item identifier. x-parser-schema-id: output_index: type: integer description: Index of the output item. x-parser-schema-id: content_index: type: integer description: Index of the content part. x-parser-schema-id: text: type: string description: Complete text output. x-parser-schema-id: required: - type - event_id - response_id - text x-parser-schema-id: ResponseOutputTextDoneEvent title: response.output_text.done description: Text output finished. example: |- { "const": "", "event_id": "", "response_id": "", "item_id": "", "output_index": 123, "content_index": 123, "text": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: responseOutputTextDone - &ref_42 id: responseOutputAudioTranscriptDelta payload: - name: response.output_audio_transcript.delta description: Streaming transcript for generated audio. type: object properties: - name: const type: string description: response.output_audio_transcript.delta required: false - name: event_id type: string description: Server-generated event ID. required: true - name: response_id type: string description: Response identifier. required: true - name: item_id type: string description: Item identifier. required: false - name: output_index type: integer description: Index of the output item. required: false - name: content_index type: integer description: Index of the content part. required: false - name: delta type: string description: Transcript chunk. required: true headers: [] jsonPayloadSchema: type: object properties: type: const: response.output_audio_transcript.delta x-parser-schema-id: event_id: type: string description: Server-generated event ID. x-parser-schema-id: response_id: type: string description: Response identifier. x-parser-schema-id: item_id: type: string description: Item identifier. x-parser-schema-id: output_index: type: integer description: Index of the output item. x-parser-schema-id: content_index: type: integer description: Index of the content part. x-parser-schema-id: delta: type: string description: Transcript chunk. x-parser-schema-id: required: - type - event_id - response_id - delta x-parser-schema-id: ResponseOutputAudioTranscriptDeltaEvent title: response.output_audio_transcript.delta description: Streaming transcript for generated audio. example: |- { "const": "", "event_id": "", "response_id": "", "item_id": "", "output_index": 123, "content_index": 123, "delta": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: responseOutputAudioTranscriptDelta - &ref_43 id: responseOutputAudioTranscriptDone payload: - name: response.output_audio_transcript.done description: Final transcript for generated audio. type: object properties: - name: const type: string description: response.output_audio_transcript.done required: false - name: event_id type: string description: Server-generated event ID. required: true - name: response_id type: string description: Response identifier. required: true - name: item_id type: string description: Item identifier. required: false - name: output_index type: integer description: Index of the output item. required: false - name: content_index type: integer description: Index of the content part. required: false - name: transcript type: string description: Complete transcript. required: true headers: [] jsonPayloadSchema: type: object properties: type: const: response.output_audio_transcript.done x-parser-schema-id: event_id: type: string description: Server-generated event ID. x-parser-schema-id: response_id: type: string description: Response identifier. x-parser-schema-id: item_id: type: string description: Item identifier. x-parser-schema-id: output_index: type: integer description: Index of the output item. x-parser-schema-id: content_index: type: integer description: Index of the content part. x-parser-schema-id: transcript: type: string description: Complete transcript. x-parser-schema-id: required: - type - event_id - response_id - transcript x-parser-schema-id: ResponseOutputAudioTranscriptDoneEvent title: response.output_audio_transcript.done description: Final transcript for generated audio. example: |- { "const": "", "event_id": "", "response_id": "", "item_id": "", "output_index": 123, "content_index": 123, "transcript": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: responseOutputAudioTranscriptDone - &ref_44 id: responseOutputAudioDone payload: - name: response.output_audio.done description: Audio output for a content part finished. type: object properties: - name: const type: string description: response.output_audio.done required: false - name: event_id type: string description: Server-generated event ID. required: true - name: response_id type: string description: Response identifier. required: true - name: item_id type: string description: Item identifier. required: false - name: output_index type: integer description: Index of the output item. required: false - name: content_index type: integer description: Index of the content part. required: false headers: [] jsonPayloadSchema: type: object properties: type: const: response.output_audio.done x-parser-schema-id: event_id: type: string description: Server-generated event ID. x-parser-schema-id: response_id: type: string description: Response identifier. x-parser-schema-id: item_id: type: string description: Item identifier. x-parser-schema-id: output_index: type: integer description: Index of the output item. x-parser-schema-id: content_index: type: integer description: Index of the content part. x-parser-schema-id: required: - type - event_id - response_id x-parser-schema-id: ResponseOutputAudioDoneEvent title: response.output_audio.done description: Audio output for a content part finished. example: |- { "const": "", "event_id": "", "response_id": "", "item_id": "", "output_index": 123, "content_index": 123 } bindings: [] extensions: - id: x-parser-unique-object-id value: responseOutputAudioDone - &ref_45 id: responseFunctionCallArgumentsDelta payload: - name: response.function_call_arguments.delta description: Streaming function call arguments. type: object properties: - name: const type: string description: response.function_call_arguments.delta required: false - name: event_id type: string description: Server-generated event ID. required: true - name: response_id type: string description: Response identifier. required: true - name: item_id type: string description: Item identifier. required: false - name: output_index type: integer description: Index of the output item. required: false - name: content_index type: integer description: Index of the content part. required: false - name: delta type: string description: Arguments chunk (JSON string fragment). required: true headers: [] jsonPayloadSchema: type: object properties: type: const: response.function_call_arguments.delta x-parser-schema-id: event_id: type: string description: Server-generated event ID. x-parser-schema-id: response_id: type: string description: Response identifier. x-parser-schema-id: item_id: type: string description: Item identifier. x-parser-schema-id: output_index: type: integer description: Index of the output item. x-parser-schema-id: content_index: type: integer description: Index of the content part. x-parser-schema-id: delta: type: string description: Arguments chunk (JSON string fragment). x-parser-schema-id: required: - type - event_id - response_id - delta x-parser-schema-id: ResponseFunctionCallArgumentsDeltaEvent title: response.function_call_arguments.delta description: Streaming function call arguments. example: |- { "const": "", "event_id": "", "response_id": "", "item_id": "", "output_index": 123, "content_index": 123, "delta": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: responseFunctionCallArgumentsDelta - &ref_46 id: responseFunctionCallArgumentsDone payload: - name: response.function_call_arguments.done description: Function call arguments finished. type: object properties: - name: const type: string description: response.function_call_arguments.done required: false - name: event_id type: string description: Server-generated event ID. required: true - name: response_id type: string description: Response identifier. required: true - name: item_id type: string description: Item identifier. required: false - name: output_index type: integer description: Index of the output item. required: false - name: content_index type: integer description: Index of the content part. required: false - name: arguments type: string description: Complete function call arguments (JSON string). required: true headers: [] jsonPayloadSchema: type: object properties: type: const: response.function_call_arguments.done x-parser-schema-id: event_id: type: string description: Server-generated event ID. x-parser-schema-id: response_id: type: string description: Response identifier. x-parser-schema-id: item_id: type: string description: Item identifier. x-parser-schema-id: output_index: type: integer description: Index of the output item. x-parser-schema-id: content_index: type: integer description: Index of the content part. x-parser-schema-id: arguments: type: string description: Complete function call arguments (JSON string). x-parser-schema-id: required: - type - event_id - response_id - arguments x-parser-schema-id: ResponseFunctionCallArgumentsDoneEvent title: response.function_call_arguments.done description: Function call arguments finished. example: |- { "const": "", "event_id": "", "response_id": "", "item_id": "", "output_index": 123, "content_index": 123, "arguments": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: responseFunctionCallArgumentsDone - &ref_47 id: inputAudioSpeechStarted payload: - name: input_audio_buffer.speech_started description: Voice activity detected — user started speaking. type: object properties: - name: const type: string description: input_audio_buffer.speech_started required: false - name: event_id type: string description: Server-generated event ID. required: true - name: audio_start_ms type: integer description: >- Millisecond offset in the audio stream where speech was detected. required: false - name: item_id type: string description: The conversation item ID associated with this speech segment. required: true headers: [] jsonPayloadSchema: type: object properties: type: const: input_audio_buffer.speech_started x-parser-schema-id: event_id: type: string description: Server-generated event ID. x-parser-schema-id: audio_start_ms: type: integer description: >- Millisecond offset in the audio stream where speech was detected. x-parser-schema-id: item_id: type: string description: The conversation item ID associated with this speech segment. x-parser-schema-id: required: - type - event_id - item_id x-parser-schema-id: InputAudioSpeechStartedEvent title: input_audio_buffer.speech_started description: Voice activity detected — user started speaking. example: |- { "const": "", "event_id": "", "audio_start_ms": 123, "item_id": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: inputAudioSpeechStarted - &ref_48 id: inputAudioSpeechStopped payload: - name: input_audio_buffer.speech_stopped description: Voice activity ended — user stopped speaking. type: object properties: - name: const type: string description: input_audio_buffer.speech_stopped required: false - name: event_id type: string description: Server-generated event ID. required: true - name: audio_end_ms type: integer description: Millisecond offset in the audio stream where speech ended. required: false - name: item_id type: string description: The conversation item ID associated with this speech segment. required: true headers: [] jsonPayloadSchema: type: object properties: type: const: input_audio_buffer.speech_stopped x-parser-schema-id: event_id: type: string description: Server-generated event ID. x-parser-schema-id: audio_end_ms: type: integer description: Millisecond offset in the audio stream where speech ended. x-parser-schema-id: item_id: type: string description: The conversation item ID associated with this speech segment. x-parser-schema-id: required: - type - event_id - item_id x-parser-schema-id: InputAudioSpeechStoppedEvent title: input_audio_buffer.speech_stopped description: Voice activity ended — user stopped speaking. example: |- { "const": "", "event_id": "", "audio_end_ms": 123, "item_id": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: inputAudioSpeechStopped - &ref_49 id: inputAudioCommitted payload: - name: input_audio_buffer.committed description: Buffered audio was committed as a conversation item. type: object properties: - name: const type: string description: input_audio_buffer.committed required: false - name: event_id type: string description: Server-generated event ID. required: true - name: previous_item_id type: &ref_9 - string - 'null' description: The ID of the preceding conversation item, or null. required: false - name: item_id type: string description: The new conversation item ID for the committed audio. required: true headers: [] jsonPayloadSchema: type: object properties: type: const: input_audio_buffer.committed x-parser-schema-id: event_id: type: string description: Server-generated event ID. x-parser-schema-id: previous_item_id: type: *ref_9 description: The ID of the preceding conversation item, or null. x-parser-schema-id: item_id: type: string description: The new conversation item ID for the committed audio. x-parser-schema-id: required: - type - event_id - item_id x-parser-schema-id: InputAudioCommittedEvent title: input_audio_buffer.committed description: Buffered audio was committed as a conversation item. example: |- { "const": "", "event_id": "", "item_id": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: inputAudioCommitted - &ref_50 id: inputAudioCleared payload: - name: input_audio_buffer.cleared description: Input audio buffer was cleared. type: object properties: - name: const type: string description: input_audio_buffer.cleared required: false - name: event_id type: string required: true headers: [] jsonPayloadSchema: type: object properties: type: const: input_audio_buffer.cleared x-parser-schema-id: event_id: type: string x-parser-schema-id: required: - type - event_id x-parser-schema-id: TypeOnlyInputAudioClearedEvent title: input_audio_buffer.cleared description: Input audio buffer was cleared. example: |- { "const": "", "event_id": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: inputAudioCleared - &ref_51 id: inputAudioTimeoutTriggered payload: - name: input_audio_buffer.timeout_triggered description: An idle timeout was triggered on the input buffer. type: object properties: - name: const type: string description: input_audio_buffer.timeout_triggered required: false - name: event_id type: string required: true headers: [] jsonPayloadSchema: type: object properties: type: const: input_audio_buffer.timeout_triggered x-parser-schema-id: event_id: type: string x-parser-schema-id: required: - type - event_id x-parser-schema-id: TypeOnlyInputAudioTimeoutTriggeredEvent title: input_audio_buffer.timeout_triggered description: An idle timeout was triggered on the input buffer. example: |- { "const": "", "event_id": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: inputAudioTimeoutTriggered - &ref_52 id: outputAudioStarted payload: - name: output_audio_buffer.started description: Server started sending output audio. type: object properties: - name: const type: string description: output_audio_buffer.started required: false - name: event_id type: string required: true headers: [] jsonPayloadSchema: type: object properties: type: const: output_audio_buffer.started x-parser-schema-id: event_id: type: string x-parser-schema-id: required: - type - event_id x-parser-schema-id: TypeOnlyOutputAudioStartedEvent title: output_audio_buffer.started description: Server started sending output audio. example: |- { "const": "", "event_id": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: outputAudioStarted - &ref_53 id: outputAudioStopped payload: - name: output_audio_buffer.stopped description: Server stopped sending output audio. type: object properties: - name: const type: string description: output_audio_buffer.stopped required: false - name: event_id type: string required: true headers: [] jsonPayloadSchema: type: object properties: type: const: output_audio_buffer.stopped x-parser-schema-id: event_id: type: string x-parser-schema-id: required: - type - event_id x-parser-schema-id: TypeOnlyOutputAudioStoppedEvent title: output_audio_buffer.stopped description: Server stopped sending output audio. example: |- { "const": "", "event_id": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: outputAudioStopped - &ref_54 id: outputAudioCleared payload: - name: output_audio_buffer.cleared description: Output audio buffer was cleared. type: object properties: - name: const type: string description: output_audio_buffer.cleared required: false - name: event_id type: string required: true headers: [] jsonPayloadSchema: type: object properties: type: const: output_audio_buffer.cleared x-parser-schema-id: event_id: type: string x-parser-schema-id: required: - type - event_id x-parser-schema-id: TypeOnlyOutputAudioClearedEvent title: output_audio_buffer.cleared description: Output audio buffer was cleared. example: |- { "const": "", "event_id": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: outputAudioCleared - &ref_55 id: rateLimitsUpdated payload: - name: rate_limits.updated description: Reports current rate limit state. type: object properties: - name: const type: string description: rate_limits.updated required: false - name: event_id type: string required: true headers: [] jsonPayloadSchema: type: object properties: type: const: rate_limits.updated x-parser-schema-id: event_id: type: string x-parser-schema-id: required: - type - event_id x-parser-schema-id: TypeOnlyRateLimitsUpdatedEvent title: rate_limits.updated description: Reports current rate limit state. example: |- { "const": "", "event_id": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: rateLimitsUpdated bindings: [] extensions: *ref_10 sendOperations: - *ref_11 receiveOperations: - *ref_12 sendMessages: - *ref_13 - *ref_14 - *ref_15 - *ref_16 - *ref_17 - *ref_18 - *ref_19 - *ref_20 - *ref_21 - *ref_22 - *ref_23 receiveMessages: - *ref_24 - *ref_25 - *ref_26 - *ref_27 - *ref_28 - *ref_29 - *ref_30 - *ref_31 - *ref_32 - *ref_33 - *ref_34 - *ref_35 - *ref_36 - *ref_37 - *ref_38 - *ref_39 - *ref_40 - *ref_41 - *ref_42 - *ref_43 - *ref_44 - *ref_45 - *ref_46 - *ref_47 - *ref_48 - *ref_49 - *ref_50 - *ref_51 - *ref_52 - *ref_53 - *ref_54 - *ref_55 extensions: - id: x-parser-unique-object-id value: realtimeStream securitySchemes: - id: bearerAuth name: bearerAuth type: http description: >- Use your API key for authentication. See [Authentication](/api-reference/introduction) for details. scheme: bearer bearerFormat: JWT extensions: [] ````