Transcribe audio (WebSocket)

Connect and authenticate

Connect to wss://api.inworld.ai/stt/v1/transcribe:streamBidirectional. Server-side clients send Authorization: Basic <INWORLD_API_KEY> on the WebSocket upgrade request. Use the Base64 credentials copied from the Portal as-is. Do not Base64-encode them again.

const WebSocket = require('ws');
const ws = new WebSocket(
  'wss://api.inworld.ai/stt/v1/transcribe:streamBidirectional',
  { headers: { Authorization: `Basic ${process.env.INWORLD_API_KEY}` } }
);

This uses the Node.js ws package. For browser or mobile clients, keep the API key on your server. Either relay audio through an authenticated backend connection or connect directly using a one-time token created by your backend.

The browser's native WebSocket constructor cannot set an Authorization header. For a direct connection, pass the token in the bearer_ WebSocket subprotocol:

// Your authenticated backend endpoint mints the token; never expose the API key.
// See the one-time-token guide for the backend handler and user authorization.
const response = await fetch('/api/inworld-token', {
  method: 'POST',
  credentials: 'same-origin',
  cache: 'no-store',
});
if (!response.ok) throw new Error(`Token request failed (${response.status})`);
const { accessToken } = await response.json();
const ws = new WebSocket(
  'wss://api.inworld.ai/stt/v1/transcribe:streamBidirectional',
  ['bearer_' + accessToken],
);

Mint a fresh token for every connection attempt, including retries and reconnects. Do not put tokens in URL query parameters. Clients that can set headers may send Authorization: Bearer <accessToken> instead. Follow your application's CSRF requirements when calling the backend token endpoint. See the one-time-token guide for the complete backend flow.

Handle connection and upgrade errors as well as JSON error messages after connection. A failed handshake may never produce a transcription or JSON error frame. Report the failure to the caller instead of waiting indefinitely for a transcript.

Streaming message sequence

Send each client message below as a separate JSON text frame. Read responses while you send audio; they can arrive at the same time.

1. Configure the stream first:

   {"transcribeConfig":{"modelId":"inworld/inworld-stt-1","audioEncoding":"LINEAR16","sampleRateHertz":16000,"numberOfChannels":1}}

2. Send audio chunks. Replace the placeholder with Base64-encoded raw PCM bytes, or send the bytes as binary messages instead — see Send audio as binary frames:

   {"audioChunk":{"content":"<base64-encoded PCM>"}}

3. Receive interim transcripts automatically. Each interim replaces the previous interim for the current turn; it is not a text delta:

   {"result":{"transcription":{"transcript":"Please open","isFinal":false}}}
   {"result":{"transcription":{"transcript":"Please open the door","isFinal":false}}}

4. Commit each final once, then clear the interim. Later audio starts a new turn:

   {"result":{"transcription":{"transcript":"Please open the door.","isFinal":true}}}

5. To finish sending audio, flush your last partial audio chunk and send:

   {"closeStream":{}}

Keep reading until the server closes the stream, including any final transcripts and result.usage. A final transcript ends a turn, not necessarily the stream. Do not close the socket immediately after sending the control message.

Send audio as binary frames

Base64 makes every audio chunk about a third larger, and audio barely compresses, so connection-level compression does not win it back. To send the raw bytes instead, add "audioProtocol": "BINARY" to the first message, next to transcribeConfig:

{"transcribeConfig":{"modelId":"inworld/inworld-stt-1","audioEncoding":"LINEAR16","sampleRateHertz":16000,"numberOfChannels":1},"audioProtocol":"BINARY"}

Then send each audio chunk as a binary WebSocket message holding the raw PCM bytes, with no JSON around them. Everything else is unchanged: endTurn and closeStream are still JSON text messages, and every response is still JSON.

ws.send(JSON.stringify({
  transcribeConfig: { modelId: 'inworld/inworld-stt-1', audioEncoding: 'LINEAR16', sampleRateHertz: 16000, numberOfChannels: 1 },
  audioProtocol: 'BINARY',
}));
ws.send(pcmChunk);                                // a Buffer or ArrayBuffer of PCM16 samples
ws.send(JSON.stringify({ closeStream: {} }));     // control messages stay JSON

• The selection is made once, on the first message, and holds for the connection. Omitting audioProtocol, or sending "JSON", keeps base64 audio. Selecting again later is refused with an error; open a new connection to switch.
• audioProtocol takes the same values as TTS's responseProtocol. For STT it changes only what you send; responses are JSON either way.
• A binary message sent without the BINARY selection is refused with an error frame, and the connection stays open. An empty binary message is ignored.
• Each WebSocket message is limited to 32 MiB.

Turn boundaries and shutdown

At shutdown, send closeStream once. Do not send endTurn first for the same audio. Repeated finalization of an empty or silent turn can produce unwanted transcripts. With automatic turn detection, pauses can already produce finals; listen for them without sending an extra endTurn for each pause.

endTurn alone does not finish the session or request final usage. Disconnecting can lose pending results and usage. Set a client-side shutdown timeout. If it expires, report the transcription as incomplete; do not silently treat a trailing interim as final. The timing of results and finalization varies with audio and model processing. A short quiet period does not mean transcription is complete.

Voice activity events

Speech events are separate from transcription results. They do not finalize a transcript. Example wire messages:

{"result":{"speechStarted":{"startTimeMs":0,"confidence":0}}}
{"result":{"speechStopped":{"silenceDurationMs":150}}}

speechStopped.silenceDurationMs describes silence when that event was emitted. It is not a continuously updated idle timer. Use transcription.isFinal to commit text. For an application idle-stop policy, maintain your own timer and reset it on speech activity and transcription results. See Turn detection.

Audio capture and runnable examples

Use signed 16-bit little-endian PCM (LINEAR16). Streaming does not accept MediaRecorder WebM/Opus or Ogg/Opus chunks. Convert captured Float32 samples to PCM16, and set sampleRateHertz to the actual rate of the samples you send. 16 kHz mono is a convenient starting point; 48 kHz PCM input is also resampled by the service. Changing the declared rate does not resample the audio.

The samples send 100 ms per chunk: 1,600 samples or 3,200 bytes at 16 kHz mono, before Base64 encoding (or exactly that, sent as a binary message). This is an example chunk size, not a protocol limit. Flush the final shorter chunk before sending closeStream.

For browser capture with an AudioWorklet, keep the processing graph connected to the destination through a zero-gain node, and wait for the worklet to flush its buffered samples before stopping the stream. Release microphone tracks and the audio context on completion or failure.

Start with the Node.js samples or Python samples. Both include file and microphone clients. The sample PCM audio is raw 16 kHz mono PCM16, so you can verify authentication, streaming and shutdown before adding microphone capture.

{
  "transcribeConfig": {
    "modelId": "inworld/inworld-stt-1",
    "audioEncoding": "LINEAR16",
    "sampleRateHertz": 16000,
    "language": "en"
  }
}

{
  "audioChunk": {
    "content": "<BASE64_AUDIO>"
  }
}

"string"

{
  "endTurn": {}
}

{
  "closeStream": {}
}

{
  "result": {
    "transcription": {
      "transcript": "Open the door and let me in, please.",
      "isFinal": true,
      "wordTimestamps": [],
      "voiceProfile": null,
      "silenceDurationMs": 0
    }
  }
}

{
  "result": {
    "usage": {
      "transcribedAudioMs": 2400,
      "modelId": "inworld/inworld-stt-1"
    }
  }
}

{
  "result": {
    "speechStarted": {
      "startTimeMs": 0,
      "confidence": 0
    }
  }
}

{
  "result": {
    "speechStopped": {
      "silenceDurationMs": 750
    }
  }
}