> ## Documentation Index > Fetch the complete documentation index at: https://docs.inworld.ai/llms.txt > Use this file to discover all available pages before exploring further. # Transcribe audio (WebSocket) > Stream audio and receive speech-to-text results in real time over a bidirectional WebSocket connection. > > The API returns recognized text chunks as they become available. Audio chunks are expected to be part of a single voice input. Use it for live conversations, microphone input, or other streaming audio sources. > > To use the API: > - Send `transcribeConfig` first to set up the session: model, language, audio encoding, and other settings. > - Stream `audioChunk` messages containing raw audio bytes. > - Read `transcription` results, including interim (partial) and final results, as they arrive. > - Listen for `speechStarted` and `speechStopped` to detect changes in voice activity. > - Optionally send `endTurn` to signal the end of a speaker's turn. > - Send `closeStream` when done. ## Connect and authenticate Connect to `wss://api.inworld.ai/stt/v1/transcribe:streamBidirectional`. Server-side clients send `Authorization: Basic ` on the WebSocket upgrade request. Use the Base64 credentials copied from the Portal as-is. Do not Base64-encode them again. ```javascript const WebSocket = require('ws'); const ws = new WebSocket( 'wss://api.inworld.ai/stt/v1/transcribe:streamBidirectional', { headers: { Authorization: `Basic ${process.env.INWORLD_API_KEY}` } } ); ``` This uses the Node.js `ws` package. For browser or mobile clients, keep the API key on your server. Either relay audio through an authenticated backend connection or connect directly using a [one-time token](https://docs.inworld.ai/portal/ephemeral-tokens.md) created by your backend. The browser's native `WebSocket` constructor cannot set an Authorization header. For a direct connection, pass the token in the `bearer_` WebSocket subprotocol: ```javascript // Your authenticated backend endpoint mints the token; never expose the API key. // See the one-time-token guide for the backend handler and user authorization. const response = await fetch('/api/inworld-token', { method: 'POST', credentials: 'same-origin', cache: 'no-store', }); if (!response.ok) throw new Error(`Token request failed (${response.status})`); const { accessToken } = await response.json(); const ws = new WebSocket( 'wss://api.inworld.ai/stt/v1/transcribe:streamBidirectional', ['bearer_' + accessToken], ); ``` Mint a fresh token for every connection attempt, including retries and reconnects. Do not put tokens in URL query parameters. Clients that can set headers may send `Authorization: Bearer ` instead. Follow your application's CSRF requirements when calling the backend token endpoint. See the [one-time-token guide](https://docs.inworld.ai/portal/ephemeral-tokens.md#connect-a-client-through-your-backend) for the complete backend flow. Handle connection and upgrade errors as well as JSON error messages after connection. A failed handshake may never produce a transcription or JSON error frame. Report the failure to the caller instead of waiting indefinitely for a transcript. ## Streaming message sequence Send each client message below as a separate JSON text frame. Read responses while you send audio; they can arrive at the same time. 1. Configure the stream first: ```json {"transcribeConfig":{"modelId":"inworld/inworld-stt-1","audioEncoding":"LINEAR16","sampleRateHertz":16000,"numberOfChannels":1}} ``` 2. Send audio chunks. Replace the placeholder with Base64-encoded raw PCM bytes, or send the bytes as binary messages instead — see [Send audio as binary frames](#send-audio-as-binary-frames): ```json {"audioChunk":{"content":""}} ``` 3. Receive interim transcripts automatically. Each interim replaces the previous interim for the current turn; it is not a text delta: ```json {"result":{"transcription":{"transcript":"Please open","isFinal":false}}} {"result":{"transcription":{"transcript":"Please open the door","isFinal":false}}} ``` 4. Commit each final once, then clear the interim. Later audio starts a new turn: ```json {"result":{"transcription":{"transcript":"Please open the door.","isFinal":true}}} ``` 5. To finish sending audio, flush your last partial audio chunk and send: ```json {"closeStream":{}} ``` Keep reading until the server closes the stream, including any final transcripts and `result.usage`. A final transcript ends a turn, not necessarily the stream. Do not close the socket immediately after sending the control message. ## Send audio as binary frames Base64 makes every audio chunk about a third larger, and audio barely compresses, so connection-level compression does not win it back. To send the raw bytes instead, add `"audioProtocol": "BINARY"` to the first message, next to `transcribeConfig`: ```json {"transcribeConfig":{"modelId":"inworld/inworld-stt-1","audioEncoding":"LINEAR16","sampleRateHertz":16000,"numberOfChannels":1},"audioProtocol":"BINARY"} ``` Then send each audio chunk as a **binary** WebSocket message holding the raw PCM bytes, with no JSON around them. Everything else is unchanged: `endTurn` and `closeStream` are still JSON text messages, and every response is still JSON. ```javascript ws.send(JSON.stringify({ transcribeConfig: { modelId: 'inworld/inworld-stt-1', audioEncoding: 'LINEAR16', sampleRateHertz: 16000, numberOfChannels: 1 }, audioProtocol: 'BINARY', })); ws.send(pcmChunk); // a Buffer or ArrayBuffer of PCM16 samples ws.send(JSON.stringify({ closeStream: {} })); // control messages stay JSON ``` - The selection is made once, on the first message, and holds for the connection. Omitting `audioProtocol`, or sending `"JSON"`, keeps base64 audio. Selecting again later is refused with an error; open a new connection to switch. - `audioProtocol` takes the same values as TTS's `responseProtocol`. For STT it changes only what you send; responses are JSON either way. - A binary message sent without the `BINARY` selection is refused with an error frame, and the connection stays open. An empty binary message is ignored. - Each WebSocket message is limited to 32 MiB. ## Turn boundaries and shutdown | Client message | Use | | :--- | :--- | | `{"endTurn":{}}` | Finalize the current turn while keeping the session open for more audio | | `{"closeStream":{}}` | Finish audio input, finalize pending audio, and let the server drain and close the session | At shutdown, send `closeStream` once. Do not send `endTurn` first for the same audio. Repeated finalization of an empty or silent turn can produce unwanted transcripts. With automatic turn detection, pauses can already produce finals; listen for them without sending an extra `endTurn` for each pause. `endTurn` alone does not finish the session or request final usage. Disconnecting can lose pending results and usage. Set a client-side shutdown timeout. If it expires, report the transcription as incomplete; do not silently treat a trailing interim as final. The timing of results and finalization varies with audio and model processing. A short quiet period does not mean transcription is complete. ## Voice activity events Speech events are separate from transcription results. They do not finalize a transcript. Example wire messages: ```json {"result":{"speechStarted":{"startTimeMs":0,"confidence":0}}} {"result":{"speechStopped":{"silenceDurationMs":150}}} ``` `speechStopped.silenceDurationMs` describes silence when that event was emitted. It is not a continuously updated idle timer. Use `transcription.isFinal` to commit text. For an application idle-stop policy, maintain your own timer and reset it on speech activity and transcription results. See [Turn detection](https://docs.inworld.ai/stt/turn-detection.md). ## Audio capture and runnable examples Use signed 16-bit little-endian PCM (`LINEAR16`). Streaming does not accept `MediaRecorder` WebM/Opus or Ogg/Opus chunks. Convert captured Float32 samples to PCM16, and set `sampleRateHertz` to the actual rate of the samples you send. 16 kHz mono is a convenient starting point; 48 kHz PCM input is also resampled by the service. Changing the declared rate does not resample the audio. The samples send 100 ms per chunk: 1,600 samples or 3,200 bytes at 16 kHz mono, before Base64 encoding (or exactly that, sent as a binary message). This is an example chunk size, not a protocol limit. Flush the final shorter chunk before sending `closeStream`. For browser capture with an AudioWorklet, keep the processing graph connected to the destination through a zero-gain node, and wait for the worklet to flush its buffered samples before stopping the stream. Release microphone tracks and the audio context on completion or failure. Start with the [Node.js samples](https://github.com/inworld-ai/inworld-api-examples/tree/main/stt/js) or [Python samples](https://github.com/inworld-ai/inworld-api-examples/tree/main/stt/python). Both include file and microphone clients. The [sample PCM audio](https://github.com/inworld-ai/inworld-api-examples/blob/main/stt/tests-data/audio/test-pcm-audio.pcm) is raw 16 kHz mono PCM16, so you can verify authentication, streaming and shutdown before adding microphone capture. ## API reference **Channel:** `WSS wss://api.inworld.ai/stt/v1/transcribe:streamBidirectional` ### Client messages #### transcribeConfig Configure the transcription session. Must be the first message sent. The config goes under a top-level `transcribeConfig` key (see the example) — an unwrapped config is rejected and the socket closes without an error frame. Beside it, the optional `audioProtocol` selects how audio is sent for the whole connection; it is a key of this message, not a config field. See [Send audio as binary frames](https://docs.inworld.ai/api-reference/sttAPI/speechtotext/transcribe-stream-websocket.md#send-audio-as-binary-frames). - `transcribeConfig` (object; required) - `modelId` (string; required) — The identifier of the model to use for transcription. Format: "{provider}/{model-name}". Available models: - `inworld/inworld-stt-1` — Inworld first-party See [STT Introduction](https://docs.inworld.ai/stt/overview.md) for the full model catalogue. - `audioEncoding` (enum; required; default: "AUDIO_ENCODING_UNSPECIFIED"; options: "AUDIO_ENCODING_UNSPECIFIED", "AUTO_DETECT", "LINEAR16", "MP3", "OGG_OPUS", "FLAC") — Supported audio encoding formats. - `AUDIO_ENCODING_UNSPECIFIED`: Not specified. Will return an error. - `AUTO_DETECT`: Automatically detect audio encoding from the audio header. - `LINEAR16`: Uncompressed 16-bit signed little-endian samples (Linear PCM). - `MP3`: MP3 audio. Not supported for streaming transcription. - `OGG_OPUS`: Opus encoded audio wrapped in an OGG container. Not supported for streaming transcription. - `FLAC`: FLAC encoded audio. Lossless format. Not supported for streaming transcription. - `language` (string) — Language hint in ISO 639 format (e.g., "en", "ja"). Biases the model toward the specified language during automatic language detection. BCP-47 codes (e.g., "en-US") are also accepted and converted to the base language code. The hint additionally constrains the output script for English, Chinese, Cantonese, Japanese, Korean, Russian, and Hindi (e.g. selecting `en` keeps output in Latin script). See [Language Support](https://docs.inworld.ai/stt/languages.md) for the full list of supported languages. - `sampleRateHertz` (integer) — Sample rate of the audio data in Hertz. Required when the sample rate cannot be inferred from the audio header (e.g., raw PCM streams). Default: 16000. - `numberOfChannels` (integer) — Number of channels in the audio data. Required when the number of channels cannot be inferred from the audio header (e.g., raw PCM streams). Default: 1. - `inactivityTimeoutSeconds` (integer) — Provider-specific inactivity setting. Not applied by inworld/inworld-stt-1; implement application idle-stop timers on the client. - `endOfTurnConfidenceThreshold` (number) — Confidence threshold for end-of-turn prediction. Higher values reduce false-positives. Range: [0.0, 1.0]. Default: 0.4. See the [Turn Detection guide](https://docs.inworld.ai/stt/turn-detection.md) for tuning guidance. - `prompts` (string[]) — Custom vocabulary / key terms. An array of context strings (names, jargon, acronyms) that bias the model toward recognizing these terms. This is a soft bias that helps with ambiguous or uncommon words; it is not a hard keyword lock and does not force exact output. Maximum 100 prompts per request. Allowed characters: letters and digits in any script, spaces, and the punctuation . , ' ( ) : ! ? ; - (straight apostrophes and hyphens only). Any other character (such as &, @, #, _, curly quotes, or en/em dashes) rejects the entire request with INVALID_ARGUMENT (code 3), naming the offending character. Short, targeted lists work best: the more prompts you send, the weaker the bias on each, and a long list is likely to be ignored. - `includeWordTimestamps` (boolean) — If true, includes per-word timing information in the response. - `enableSpeakerDiarization` (boolean) — Labels transcribed words with a per-stream speaker identifier. Experimental for `inworld/inworld-stt-1` — speaker attribution quality is still improving and speakers may occasionally be misattributed. Set it together with `includeWordTimestamps`; speaker labels are returned on `wordTimestamps[].speaker`, and some words may arrive without a label (treat those as unattributed). See the [Speaker Diarization guide](https://docs.inworld.ai/stt/speaker-diarization.md). - `inworldSttV1Config` (object) — Configuration for Inworld STT 1 models. Set `vadThreshold` to 0 to disable server-side turn detection and control turn boundaries manually via `endTurn` — see the [Turn Detection guide](https://docs.inworld.ai/stt/turn-detection.md). - `minEndOfTurnSilenceWhenConfident` (integer) — Minimum trailing silence before checking for turn completion, in milliseconds. Applies to streaming. Keep this less than or equal to maxTurnSilence; the minimum is checked before the maximum-silence fallback. - `vadThreshold` (number) — Voice activity detection threshold for streaming. Range: [0.0, 1.0]. Default: 0.15. Set to 0 for manual turn control. - `maxTurnSilence` (integer; default: 300) — Trailing silence that ends the turn even if end-of-turn confidence is insufficient, once minEndOfTurnSilenceWhenConfident has been reached, in milliseconds. Applies to streaming. This is not a transcript-delivery deadline. - `voiceProfileConfig` (object) — Configuration for voice profile detection. - `enableVoiceProfile` (boolean; required) — Enables voice profile feature for this request or stream. - `topN` (integer) — Number of top labels from each class to return. Default: 10. - `audioProtocol` (enum; options: "PROTOCOL_UNSPECIFIED", "JSON", "BINARY") — How audio is sent on this connection, chosen once, on this first message. `BINARY`: audio may be sent as binary WebSocket messages holding the raw bytes (`binaryAudio`); JSON `audioChunk` messages are still accepted. `JSON`, `PROTOCOL_UNSPECIFIED` or omitted: audio is sent as base64 in `audioChunk`. Responses are JSON either way. A later message that selects a protocol again is refused. The values are those of TTS's `responseProtocol`. **Example payload:** ```json { "transcribeConfig": { "modelId": "inworld/inworld-stt-1", "audioEncoding": "LINEAR16", "sampleRateHertz": 16000, "language": "en" } } ``` #### audioChunk Send a chunk of audio data for transcription. Must be sent after the initial transcribe config message. On the wire, wrap the fields below in a top-level `audioChunk` key (see the example). On a connection that selected `"audioProtocol": "BINARY"`, send `binaryAudio` messages instead to avoid the base64 overhead; `audioChunk` is still accepted there. - `content` (string; required) — The raw audio bytes in the encoding specified by the transcribe config's audioEncoding. **Example payload:** ```json { "audioChunk": { "content": "" } } ``` #### binaryAudio (binary frame) A binary WebSocket message carrying raw audio bytes — the same bytes an `audioChunk` would carry base64-encoded, in the `audioEncoding`, sample rate and channel count of the session's config. No JSON, envelope or length prefix: the WebSocket message boundary delimits the chunk. Accepted only after the first message selected `"audioProtocol": "BINARY"`; otherwise it is refused with an error and the connection stays open. An empty binary message is ignored. Control messages (`endTurn`, `closeStream`) remain JSON text messages, and responses remain JSON in either mode. **Example payload:** ```json "string" ``` #### endTurn Finalize the current turn while keeping the session open for more audio. Send `{"endTurn": {}}`. At stream shutdown use `closeStream` instead. **Example payload:** ```json { "endTurn": {} } ``` #### closeStream Finish audio input and finalize pending audio. Send `{"closeStream": {}}` once, without an `endTurn` immediately before it. Keep receiving final transcripts and usage until the server closes the stream. **Example payload:** ```json { "closeStream": {} } ``` ### Server messages #### transcription Transcription result streamed back as audio is processed. May be an interim (partial) result or a final result depending on the `isFinal` field. Arrives wrapped under a top-level `result` key as `result.transcription` (see the example). Each interim replaces the previous interim for the current turn; it is not a delta. Commit a final once and clear the interim. - `transcript` (string) — Full transcribed text for this segment. - `isFinal` (boolean) — Indicates whether this is a finalized result or an interim (partial) result that may be updated as more audio is processed. - `wordTimestamps` (object[]) — Per-word timing and confidence data. Only populated when `includeWordTimestamps` is enabled. - `word` (string) — The transcribed word. - `confidence` (number) — Recognition confidence score for this word. Range: [0.0, 1.0]. Not currently populated: the synchronous and streaming endpoints return 0, and asynchronous results omit the field. - `startTimeMs` (integer) — Offset from the beginning of the audio to the start of this word, in milliseconds. - `endTimeMs` (integer) — Offset from the beginning of the audio to the end of this word, in milliseconds. - `speaker` (integer) — Speaker identifier for this word when diarization is enabled. Integers are assigned per stream in order of first appearance. May be absent on some words even when diarization is enabled — treat unlabeled words as unattributed. See the [Speaker Diarization guide](https://docs.inworld.ai/stt/speaker-diarization.md). - `voiceProfile` (object,null) — Voice Profile classification results. `null` unless `voiceProfileConfig.enableVoiceProfile` is set in the transcribe config. See the [Voice Profiles guide](https://docs.inworld.ai/stt/voice-profiles.md) for the category structure. - `silenceDurationMs` (integer) — Duration of trailing silence detected for this result, in milliseconds. **Example payload:** ```json { "result": { "transcription": { "transcript": "Open the door and let me in, please.", "isFinal": true, "wordTimestamps": [], "voiceProfile": null, "silenceDurationMs": 0 } } } ``` #### usage Session usage, wrapped as `result.usage`, delivered during graceful stream completion. Keep reading after `closeStream`; `endTurn` alone does not request final session usage. Abrupt disconnection may lose this message. - `transcribedAudioMs` (integer) — The duration of the transcribed audio in milliseconds. - `modelId` (string) — The identifier of the model used for transcription. **Example payload:** ```json { "result": { "usage": { "transcribedAudioMs": 2400, "modelId": "inworld/inworld-stt-1" } } } ``` #### speechStarted Signal to indicate the start of a speaker's speech. Sent when voice activity is detected in the audio stream, wrapped under a top-level `result` key as `result.speechStarted`. - `startTimeMs` (integer) — The timestamp of the start of the speech in milliseconds. - `confidence` (number) — The confidence score of the speech detection. **Example payload:** ```json { "result": { "speechStarted": { "startTimeMs": 0, "confidence": 0 } } } ``` #### speechStopped Signal raised when STT detects silence after speech has stopped. Useful for tracking pauses and implementing custom turn-taking logic. Arrives wrapped under a top-level `result` key as `result.speechStopped`. - `silenceDurationMs` (integer) — The duration of silence detected after speech stopped, in milliseconds. **Example payload:** ```json { "result": { "speechStopped": { "silenceDurationMs": 750 } } } ```