WebSocket

Synthesize speech (WebSocket)

WSSwss://api.inworld.ai/tts/v1/voice:streamBidirectional

Generate audio from text input while managing multiple independent audio generation streams over a single WebSocket connection.

The independent audio streams each correspond to a context, identified by contextId, that maintains its own state. To use the API:

Create a context with audio generation configurations. By default, we allow up to 20 concurrent connections, with a maximum of 5 contexts per connection.
When you send text to be synthesized into audio, you can send it to a specific context (optional if there is only 1 context).
Each context maintains its own buffer that can be flushed either manually or automatically when the buffer reaches a certain threshold (see maxBufferDelayMs and bufferCharThreshold in the context configurations).
If texts are sent in full sentences phrases, it's recommended to use auto_mode which would automatically balance latency and quality of the generations.
Responses contain the contextId so you can match the audio to the request.
Close a context when it is no longer needed.

Client messages

Create Context

Create a new context with specified voice and configuration. A context is an independent conversation happening over the connection. The configurations for each context are completely separate – you can have different voice ids, models, output formats, etc. between contexts. Note: for each connection, 5 contexts is the max. If you don't need multiple contexts, you can omit the contextId in the message to use a single context connection.

Payload

createobjectrequired

Show child attributes

voiceIdstringrequired

The identifier of the voice to use for the synthesis

modelIdstringrequired

The ID of the model to use for synthesizing speech. See Models for available models.

audioConfigobject

Show child attributes

audioEncodingenum<string>default: "MP3"

The desired output format of the synthesized audio. Defaults to MP3.

LINEAR16: Uncompressed 16-bit signed little-endian samples (Linear PCM). The WAV header is included in every audio chunk.
MP3: MP3 audio.
OGG_OPUS: Opus encoded audio wrapped in an ogg container. The result will be a file which can be played natively on Android, and in browsers (at least Chrome and Firefox). The quality of the encoding is considerably higher than MP3 while using approximately the same bitrate.
ALAW: ALAW encoded audio. 8-bit companded PCM.
MULAW: MULAW encoded audio. 8-bit companded PCM.
PCM: PCM audio. Uncompressed 16-bit signed little-endian samples with no WAV header.
WAV: WAV audio. Uncompressed 16-bit signed little-endian samples. The WAV header is included in the first audio chunk only. On each flush_completed response, the next audio chunk will also start with a header.

Available options:AUDIO_ENCODING_UNSPECIFIEDLINEAR16MP3OGG_OPUSALAWMULAWPCMWAV

sampleRateHertzinteger

The synthesis sample rate (in hertz) for this audio. Accepts values within the range [8000, 48000].

When this is specified, if this is different from the voice's natural sample rate, then the audio will be converted to the desired sample rate (which might result in worse audio quality), unless the specified sample rate is not supported for the encoding chosen, in which case it will fail the request and return an error. The default is 48,000.

bitRateinteger

Bits per second of the audio. Only for compressed audio formats (MP3, OGG_OPUS). The default is 128,000.

speakingRatenumber

Speaking rate/speed, in the range [0.5, 1.5]. The default is 1.0, which is the normal native speed supported by the specific voice. We recommend using values above 0.8 to ensure high quality.

temperaturenumber

Ignored on `inworld-tts-2`. Use [`deliveryMode`](#deliveryMode) instead.

Determines the degree of randomness when sampling audio tokens to generate the response.

Defaults to 1.0. Accepts values between 0 (exclusive) and 2 (inclusive). Higher values will make the output more random and can lead to more expressive results. Lower values will make it more deterministic. If 0 is provided, the default value will be used.

For the most stable results, we recommend using the default value.

timestampTypeenum<string>default: "TIMESTAMP_TYPE_UNSPECIFIED"

Controls timestamp metadata returned with the audio. When enabled, the response includes timing arrays, which can be useful for word-highlighting, karaoke-style captions, and lipsync.

WORD: Output arrays under timestampInfo.wordAlignment (words, wordStartTimeSeconds, wordEndTimeSeconds).
CHARACTER: Output arrays under timestampInfo.characterAlignment (characters, characterStartTimeSeconds, characterEndTimeSeconds).
TIMESTAMPTYPEUNSPECIFIED: Do not compute alignment; timestamp arrays will be empty or omitted.

Latency note: Alignment adds additional computation. Enabling alignment can increase latency.

The timestamps reset per flush, either triggered manually or automatically by the server. When you receive a flushCompleted message, the timestamps for subsequent chunks will start from 0.

Available options:TIMESTAMP_TYPE_UNSPECIFIEDWORDCHARACTER

maxBufferDelayMsinteger

If set, determines the maximum time in milliseconds to buffer before starting generation. The timer starts running when the first text in the buffer is received and resets when new text arrives.

If not set or set to 0, there will be no time-based limit on the buffer. Instead, the user will need to flush the buffer or set the bufferCharThreshold to trigger audio generation.

Note that both length and timeout based flushing can be used together - if both maxBufferDelayMs and bufferCharThreshold are set, the server will flush if either condition is met.

bufferCharThresholdinteger

Defines the minimum number of characters in the buffer that would automatically trigger audio generation. This allows you to rely on automatic triggering instead of calling flush manually. If set to 0 or left undefined, the threshold defaults to 1000 to ensure stable behavior. Cannot be set to a value greater than 2000, the per-send_text maximum.

applyTextNormalizationenum<string>default: "APPLY_TEXT_NORMALIZATION_UNSPECIFIED"

When enabled, text normalization automatically expands and standardizes things like numbers, dates, times, and abbreviations before converting them to speech. For example, Dr. Smith becomes Doctor Smith, and 3/10/25 is spoken as March tenth, twenty twenty-five. Turning this off may reduce latency, but the speech output will read the text exactly as written. Defaults to automatically deciding whether to apply text normalization.

Available options:APPLY_TEXT_NORMALIZATION_UNSPECIFIEDONOFF

autoModebooleandefault: false

When enabled, the server will control flushing of buffered text to achieve minimal latency, while still maintaining high quality audio output. Recommended when texts are sent in full sentences/phrases. The default is false.

timestampTransportStrategyenum<string>default: "TIMESTAMP_TRANSPORT_STRATEGY_UNSPECIFIED"

The transport strategy of timestamps info.

TIMESTAMP_TRANSPORT_STRATEGY_UNSPECIFIED: The service will automatically decide the transport strategy.
SYNC: Timestamps will be returned in the same message as the audio data.
ASYNC: Timestamps could return in trailing message after the audio data. Use this strategy to reduce latency of the first audio chunk.

Available options:TIMESTAMP_TRANSPORT_STRATEGY_UNSPECIFIEDSYNCASYNC

languagestring

BCP-47 language tag (e.g., en-US, fr-FR, ja-JP) specifying the language that the given voice should speak the text in. If a localized voice prompt exists for the language, it will be used. When omitted, the original voice prompt will be used and the language will be auto-detected from the input text. If an invalid language code is provided, an error will be returned.

See Languages for more details.

deliveryModeenum<string>default: "DELIVERY_MODE_UNSPECIFIED"

Only supported by `inworld-tts-2`. The field is ignored on other models.

Controls how varied the output is.

DELIVERY_MODE_UNSPECIFIED: Defaults to BALANCED behavior.
STABLE: Optimizes for more consistent, predictable output.
BALANCED: Balanced between stability and diversity.
CREATIVE: Optimizes for increased emotional range and variation.

Available options:DELIVERY_MODE_UNSPECIFIEDSTABLEBALANCEDCREATIVE

contextIdstring

Optional context ID. If not provided, one will be generated

Example

{
  "create": {
    "voiceId": "Dennis",
    "modelId": "inworld-tts-2",
    "bufferCharThreshold": 100,
    "autoMode": true,
    "timestampType": "WORD",
    "timestampTransportStrategy": "ASYNC"
  },
  "contextId": "ctx-1"
}

Send Text

Send text to be synthesized for a specific context. You can only send up to 2000 characters in a single sendtext request. Text can be buffered on the server or immediately flushed by including `flushcontext` in the message.

Payload

send_textobjectrequired

Show child attributes

textstringrequired

The text to synthesize. Maximum 2000 characters per send_text request.

flush_contextobject

Flush a context and start synthesis of all accumulated text

contextIdstring

The target context for this message. Optional if only one context has been opened

Example

{
  "send_text": {
    "text": "Hello, what a wonderful day to be a text-to-speech model!",
    "flush_context": {}
  },
  "contextId": "ctx-1"
}

Flush Context

Flush a context and start synthesis of all accumulated text. The buffer also flushes on its own once accumulated text reaches buffer_char_threshold (1000 characters by default), independently of max_buffer_delay_ms.

Payload

flush_contextobjectrequired

contextIdstring

The context to flush. Optional if only one context has been opened

Example

{
  "flush_context": {},
  "contextId": "ctx-1"
}

Close Context

Close an existing context and release all of its resources. Sending a close context message is equivalent to sending a flush message right before, so all text in the buffer will be synthesized before the context is closed. Note that the session will automatically be closed after 10 minutes of inactivity across any context.

Payload

close_contextobjectrequired

contextIdstring

The context to close

Example

{
  "close_context": {},
  "contextId": "ctx-1"
}

Server messages

Context Created

Event sent when a new TTS context has been successfully created

Payload

resultobject

Show child attributes

contextIdstring

The ID of the created context

contextCreatedobject

Show child attributes

voiceIdstring

The identifier of the voice to use for the synthesis

modelIdstring

The ID of the model to use for synthesizing speech. See Models for available models.

audioConfigobject

Show child attributes

audioEncodingenum<string>default: "MP3"

The desired output format of the synthesized audio. Defaults to MP3.

LINEAR16: Uncompressed 16-bit signed little-endian samples (Linear PCM). The WAV header is included in every audio chunk.
MP3: MP3 audio.
OGG_OPUS: Opus encoded audio wrapped in an ogg container. The result will be a file which can be played natively on Android, and in browsers (at least Chrome and Firefox). The quality of the encoding is considerably higher than MP3 while using approximately the same bitrate.
ALAW: ALAW encoded audio. 8-bit companded PCM.
MULAW: MULAW encoded audio. 8-bit companded PCM.
PCM: PCM audio. Uncompressed 16-bit signed little-endian samples with no WAV header.
WAV: WAV audio. Uncompressed 16-bit signed little-endian samples. The WAV header is included in the first audio chunk only. On each flush_completed response, the next audio chunk will also start with a header.

Available options:AUDIO_ENCODING_UNSPECIFIEDLINEAR16MP3OGG_OPUSALAWMULAWPCMWAV

sampleRateHertzinteger

The synthesis sample rate (in hertz) for this audio. Accepts values within the range [8000, 48000].

bitRateinteger

Bits per second of the audio. Only for compressed audio formats (MP3, OGG_OPUS). The default is 128,000.

speakingRatenumber

Speaking rate/speed, in the range [0.5, 1.5]. The default is 1.0, which is the normal native speed supported by the specific voice. We recommend using values above 0.8 to ensure high quality.

temperaturenumber

Ignored on `inworld-tts-2`. Use [`deliveryMode`](#deliveryMode) instead.

Determines the degree of randomness when sampling audio tokens to generate the response.

For the most stable results, we recommend using the default value.

timestampTypeenum<string>default: "TIMESTAMP_TYPE_UNSPECIFIED"

Controls timestamp metadata returned with the audio. When enabled, the response includes timing arrays, which can be useful for word-highlighting, karaoke-style captions, and lipsync.

WORD: Output arrays under timestampInfo.wordAlignment (words, wordStartTimeSeconds, wordEndTimeSeconds).
CHARACTER: Output arrays under timestampInfo.characterAlignment (characters, characterStartTimeSeconds, characterEndTimeSeconds).
TIMESTAMPTYPEUNSPECIFIED: Do not compute alignment; timestamp arrays will be empty or omitted.

Latency note: Alignment adds additional computation. Enabling alignment can increase latency.

The timestamps reset per flush, either triggered manually or automatically by the server. When you receive a flushCompleted message, the timestamps for subsequent chunks will start from 0.

Available options:TIMESTAMP_TYPE_UNSPECIFIEDWORDCHARACTER

maxBufferDelayMsinteger

If set, determines the maximum time in milliseconds to buffer before starting generation. The timer starts running when the first text in the buffer is received and resets when new text arrives.

If not set or set to 0, there will be no time-based limit on the buffer. Instead, the user will need to flush the buffer or set the bufferCharThreshold to trigger audio generation.

Note that both length and timeout based flushing can be used together - if both maxBufferDelayMs and bufferCharThreshold are set, the server will flush if either condition is met.

bufferCharThresholdinteger

applyTextNormalizationenum<string>default: "APPLY_TEXT_NORMALIZATION_UNSPECIFIED"

Available options:APPLY_TEXT_NORMALIZATION_UNSPECIFIEDONOFF

autoModebooleandefault: false

timestampTransportStrategyenum<string>default: "TIMESTAMP_TRANSPORT_STRATEGY_UNSPECIFIED"

The transport strategy of timestamps info.

TIMESTAMP_TRANSPORT_STRATEGY_UNSPECIFIED: The service will automatically decide the transport strategy.
SYNC: Timestamps will be returned in the same message as the audio data.
ASYNC: Timestamps could return in trailing message after the audio data. Use this strategy to reduce latency of the first audio chunk.

Available options:TIMESTAMP_TRANSPORT_STRATEGY_UNSPECIFIEDSYNCASYNC

languagestring

See Languages for more details.

deliveryModeenum<string>default: "DELIVERY_MODE_UNSPECIFIED"

Only supported by `inworld-tts-2`. The field is ignored on other models.

Controls how varied the output is.

DELIVERY_MODE_UNSPECIFIED: Defaults to BALANCED behavior.
STABLE: Optimizes for more consistent, predictable output.
BALANCED: Balanced between stability and diversity.
CREATIVE: Optimizes for increased emotional range and variation.

Available options:DELIVERY_MODE_UNSPECIFIEDSTABLEBALANCEDCREATIVE

statusobject

Status information for gRPC responses, including any error details if applicable

Show child attributes

codeinteger

The status code, as specified by gRPC status codes.

messagestring

A short description of the error

detailsobject[]

Additional status or error details

Example

{
  "result": {
    "contextId": "ctx-1",
    "contextCreated": {
      "voiceId": "Dennis",
      "audioConfig": {
        "audioEncoding": "LINEAR16",
        "sampleRateHertz": 16000
      },
      "modelId": "inworld-tts-2",
      "timestampType": "WORD",
      "maxBufferDelayMs": 3000,
      "autoMode": true,
      "timestampTransportStrategy": "SYNC",
      "language": "en-US",
      "deliveryMode": "BALANCED"
    },
    "status": {
      "code": 0,
      "message": "",
      "details": []
    }
  }
}

Audio Chunk

Audio data chunk containing synthesized speech

Payload

resultobject

Show child attributes

contextIdstring

The context this audio chunk belongs to

audioChunkobject

Show child attributes

audioContentstring

Base64 encoded audio data

usageobject

Show child attributes

processedCharactersCountinteger

Number of characters of the input text processed so far

modelIdstring

The model used for speech synthesis

timestampInfoobject

Timestamp alignment information when alignment is enabled

Show child attributes

wordAlignmentobject

Word-level alignment when timestampType is WORD

Show child attributes

wordsstring[]

Aligned words in order

wordStartTimeSecondsnumber[]

Start time for each word in seconds from the beginning of the audio

wordEndTimeSecondsnumber[]

End time for each word in seconds from the beginning of the audio

characterAlignmentobject

Character-level alignment when timestampType is CHARACTER

Show child attributes

charactersstring[]

Aligned characters (including punctuation and spaces) in order

characterStartTimeSecondsnumber[]

Start time for each character in seconds from the beginning of the audio

characterEndTimeSecondsnumber[]

End time for each character in seconds from the beginning of the audio

statusobject

Status information for gRPC responses, including any error details if applicable

Show child attributes

codeinteger

The status code, as specified by gRPC status codes.

messagestring

A short description of the error

detailsobject[]

Additional status or error details

Example

{
  "result": {
    "contextId": "ctx-1",
    "audioChunk": {
      "audioContent": "UklGRgSYAABXQVZFZm10IBAAAAABAAEAgD4AAAB9AAACABAAZGF0YeCX=",
      "usage": {
        "processedCharactersCount": 79,
        "modelId": "inworld-tts-2"
      },
      "timestampInfo": {
        "wordAlignment": {
          "words": [
            "Hello,",
            "what",
            "a",
            "wonderful",
            "day",
            "to",
            "be",
            "a",
            "text-to-speech",
            "model."
          ],
          "wordStartTimeSeconds": [
            0.031,
            0.375,
            0.901,
            1.002,
            1.386,
            1.548,
            1.649,
            1.771,
            1.852,
            2.58
          ],
          "wordEndTimeSeconds": [
            0.355,
            0.86,
            0.921,
            1.326,
            1.528,
            1.609,
            1.71,
            1.791,
            2.539,
            2.802
          ]
        }
      },
      "status": {
        "code": 0,
        "message": "",
        "details": []
      }
    }
  }
}

Context Closed

Event sent when a context has been closed

Payload

resultobject

Show child attributes

contextIdstring

The ID of the closed context

contextClosedobject

statusobject

Status information for gRPC responses, including any error details if applicable

Show child attributes

codeinteger

The status code, as specified by gRPC status codes.

messagestring

A short description of the error

detailsobject[]

Additional status or error details

Example

{
  "result": {
    "contextId": "ctx-1",
    "contextClosed": {},
    "status": {
      "code": 0,
      "message": "",
      "details": []
    }
  }
}

Flush Completed

Event sent when speech synthesis for a flush of text is completed. Some websocket use cases require an indicator that speech synthesis for a flush of text is completed. To facilitate this, we've included an empty "flushCompleted":{} event at the end of speech synthesis for each flush. Note that the implementation currently assumes that flushes execute sequentially, so the first flushCompleted event would correspond to the first flush call made on the client side.

Payload

resultobject

Show child attributes

contextIdstring

The ID of the context for which the flush completed

flushCompletedobject

statusobject

Status information for gRPC responses, including any error details if applicable

Show child attributes

codeinteger

The status code, as specified by gRPC status codes.

messagestring

A short description of the error

detailsobject[]

Additional status or error details

Example

{
  "result": {
    "contextId": "ctx-1",
    "flushCompleted": {},
    "status": {
      "code": 0,
      "message": "",
      "details": []
    }
  }
}