Realtime TTS-2 is live. Built for realtime conversation that feels human. Read the Realtime TTS-2 announcement

Voice responses (LLM + TTS)

Generate a model answer and synthesize it with Inworld TTS in one chat completions request.

Overview

Voice responses generate an answer and speak it in one API call (LLM + TTS). Add audio to a chat completion: Inworld Inference generates the answer with your chosen model, and Inworld TTS speaks it. You receive both text and audio, with or without streaming.

Voice responses take the conversation as messages and speak the answer. For a live conversation with speech input, turn detection, and interruptions, use the Realtime API. To synthesize text you already have, use TTS.

In streaming mode, text from the selected model is sent to Inworld TTS as it is generated, and the response contains audio chunks with their transcript. In non-streaming mode, the complete audio and transcript are returned together once the response is ready.

This is ideal for:

  • Voice assistants and conversational agents
  • Real-time narration and read-aloud features
  • Accessibility-first applications
  • Any workflow where your users hear AI responses instead of (or in addition to) reading them

Choose a model

Choose a text model from inworld.ai/models and copy its API model ID. The example uses openai/gpt-5.4 for the answer and inworld-tts-2 for speech.

Generate a spoken response

Add the audio parameter to your chat completions request. Run the request on your server with the complete Base64 credential from Portal in INWORLD_API_KEY; do not encode it again. See the Inference quickstart for setup.

bash
curl --request POST \
  --url https://api.inworld.ai/v1/chat/completions \
  --header "Authorization: Basic $INWORLD_API_KEY" \
  --header 'Content-Type: application/json' \
  --data '{
    "model": "openai/gpt-5.4",
    "max_tokens": 1000,
    "stream": true,
    "audio": {
      "voice": "Dennis",
      "model": "inworld-tts-2"
    },
    "messages": [
      {"role": "user", "content": "What is the meaning of life?"}
    ]
  }'

That's it. Inworld Inference will:

  1. Generate an answer with the selected model
  2. Stream text chunks to Inworld TTS as they're generated
  3. Return both text and audio in the SSE stream

Audio Parameters

The audio object controls voice synthesis:

ParameterTypeDescription
voicestringRequired. The voice ID to use for speech synthesis (e.g., "Dennis", "Chloe"). See List Voices for all available voices.
modelstringRequired. The TTS model to use (e.g., "inworld-tts-2"). See TTS Models for available options.
formatstringOptional: "pcm16" (default), "mulaw" or "alaw". See Audio formats.
sample_rateintegerOptional: the output sample rate in Hz. See Audio formats for the rates each format accepts.
timestamp_typestringOptional: "word" or "character". See Optional timestamps.
speaking_ratenumberOptional: how fast the voice speaks, from 0.5 to 1.5. 1.0, the default, is the voice's normal pace; lower is slower. Values above 0.8 keep the best quality. A value outside the range returns 400.
temperaturenumberOptional: the TTS sampling temperature, from 0 to 2; 0 means the default, 1.0. Higher values make the delivery more varied. inworld-tts-2 ignores it (TTS temperature). This is separate from the top-level temperature, which applies to the text model. A value outside the range returns 400.
text_filtersarray of stringsOptional: up to 10 strings to remove from the answer before it is spoken, each up to 128 bytes. Each string is matched literally, not as a pattern. Use it to keep characters such as * or # out of the speech. More than 10 strings, or a longer string, returns 400.

audio.model must name an Inworld TTS model, starting with inworld-. An audio object with any other model, such as OpenAI's tts-1, returns 400.

Audio formats

Audio is always mono and headerless, in both streaming and non-streaming responses.

formatEncodingsample_rate valuesDefault rate
pcm1616-bit signed little-endian PCM8000, 16000, 22050, 24000, 32000, 44100, 4800048000
mulawG.711 µ-law, 8-bit80008000
alawG.711 A-law, 8-bit80008000

Any other format, including OpenAI's wav, mp3, flac, opus and aac, or a sample_rate the format does not accept, returns 400. Use mulaw or alaw for telephony that expects G.711 audio.

Request options with voice responses

Most chat completions options work with audio. Because the answer is spoken, these behave differently. The 400 refusals happen before any model is called, so a request is never answered without its audio:

OptionWith audio
response_format json_schema or json_objectReturns 400, streaming or not. The answer would be read aloud rather than returned as JSON. To get both, make one request for the JSON and speak the part you need with a second request or with TTS.
n greater than 1Returns 400 with stream: true. A spoken stream carries one answer.
logprobs: trueReturns 400 with stream: true.
Web search with engine exa, brave or googleWorks with stream: false: the final answer is spoken, and its url_citation annotations are kept on the message. With stream: true it returns 400; set stream to false, or use engine: "native", which can stream speech.
modalitiesOptional. ["text", "audio"], which the OpenAI SDK sends for spoken output, works: the model writes text and Inworld TTS speaks it. Leaving modalities out works the same.
toolsSupported, streaming or not. See Tool calling.

Streaming Response Format

When streaming is enabled ("stream": true), the response is delivered as Server-Sent Events (SSE). Each event is a JSON object in the data field.

When TTS is active, text is delivered through delta.audio.transcript. Audio data and its corresponding transcript are sent together via delta.audio:

json
data: {"choices":[{"delta":{"audio":{"data":"<base64-audio>","transcript":"Hello! How can I assist you today?"}},"index":0}],...}
FieldDescription
delta.audio.dataBase64-encoded audio in the requested format.
delta.audio.transcriptThe text being spoken. Use this for real-time text display.

Text and audio are chunked independently, so don't treat one transcript value as one sentence or as the exact words of the audio in the same chunk. Audio arrives in frames, and a chunk that has no new text omits transcript. With TTS-2 models (inworld-tts-2 and inworld-tts-2-flash), a transcript piece is the text sent to TTS since the previous chunk, so it can cut a sentence in two and run ahead of the audio. With older TTS models, a piece is a whole sentence, attached to its first audio chunk. In both cases, the joined transcript pieces are the full spoken text.

A reasoning model's reasoning arrives as delta.reasoning text, as in a stream without audio. It is never spoken. Tool calls arrive as delta.tool_calls (see Tool calling).

Non-Streaming Response

Without streaming ("stream": false), the full audio and transcript are returned in the message.audio object:

json
{
  "choices": [{
    "message": {
      "role": "assistant",
      "content": "",
      "audio": {
        "id": "audio_chatcmpl-xyz",
        "data": "<base64-audio>",
        "transcript": "Hello! How can I assist you today?"
      }
    },
    "finish_reason": "stop"
  }]
}

When TTS is active, message.content is empty. The full text is available in message.audio.transcript.

Choose another text model

Choose the text model for the answer separately from the TTS model and voice. Browse inworld.ai/models for model IDs. You can also use optional automatic model selection with voice responses:

bash
# Use auto model selection + TTS
curl --request POST \
  --url https://api.inworld.ai/v1/chat/completions \
  --header "Authorization: Basic $INWORLD_API_KEY" \
  --header 'Content-Type: application/json' \
  --data '{
    "model": "auto",
    "stream": true,
    "audio": {
      "voice": "Chloe",
      "model": "inworld-tts-2"
    },
    "messages": [
      {"role": "user", "content": "Tell me a short bedtime story."}
    ],
    "extra_body": {
      "sort": ["latency"]
    }
  }'

This example requests automatic model selection with latency ordering, then synthesizes the answer with Inworld TTS. Latency ordering does not guarantee that a particular request will be served by the fastest model.

Add routing when needed

You can add fallbacks or model-selection preferences to a voice-response request:

Fallbacks with voice

Configure backup models for failures before the response starts. See Fallbacks for retry behavior and streaming limits:

bash
curl --request POST \
  --url https://api.inworld.ai/v1/chat/completions \
  --header "Authorization: Basic $INWORLD_API_KEY" \
  --header 'Content-Type: application/json' \
  --data '{
    "model": "openai/gpt-5.4",
    "stream": true,
    "audio": {
      "voice": "Dennis",
      "model": "inworld-tts-2"
    },
    "messages": [
      {"role": "user", "content": "Explain quantum computing simply."}
    ],
    "extra_body": {
      "models": ["anthropic/claude-opus-4-6", "google-ai-studio/gemini-2.5-flash"]
    }
  }'

If GPT-5 fails, the request falls back to Claude or Gemini, and the same voice (Dennis) is used whichever model generates the text.

Cost-Optimized Voice Responses

Route to the cheapest model while still getting audio output:

bash
curl --request POST \
  --url https://api.inworld.ai/v1/chat/completions \
  --header "Authorization: Basic $INWORLD_API_KEY" \
  --header 'Content-Type: application/json' \
  --data '{
    "model": "auto",
    "stream": true,
    "audio": {
      "voice": "Dennis",
      "model": "inworld-tts-2"
    },
    "messages": [
      {"role": "user", "content": "What is the capital of France?"}
    ],
    "extra_body": {
      "sort": ["price", "latency"]
    }
  }'

Stream text into speech

Speech starts before the model finishes its answer. Text is cut at sentence boundaries: sentence-ending punctuation (., !, ? and their CJK equivalents) and line breaks. Commas, colons and semicolons never cut a sentence. First audio waits for the first sentence to end, so a short first sentence starts speech sooner.

How sentences are synthesized depends on the TTS model and on streaming:

RequestHow the answer is spoken
stream: true with a TTS-2 model (inworld-tts-2, inworld-tts-2-flash)The text streams into one TTS context as the model writes it, and TTS cuts it at sentence boundaries. With inworld-tts-2, the delivery stays consistent from one sentence to the next. inworld-tts-2-flash speaks each sentence on its own.
stream: true with an older TTS modelThe router cuts the text into sentences and synthesizes each one on its own.
stream: falseThe router cuts the complete answer into sentences, synthesizes each one on its own, and returns the joined audio.

To keep one consistent delivery across a long spoken answer, stream it with inworld-tts-2.

A sentence or line with no letter or digit, such as an emoji, --- or a code fence, is not spoken. To keep other characters, such as * or #, out of the speech, list them in text_filters.

Handle errors

StatusWhenWhat to do
400An audio field is invalid (an unknown format, a sample_rate the format doesn't take, a speaking_rate or temperature out of range, a model that isn't an Inworld TTS model), or the request combines audio with an option it can't use. error.message names the field.Fix the request. Don't retry it unchanged.
502Speech synthesis failed. error.type is upstream_error and error.message is Text-to-speech synthesis failed.Retry the request.

A failed synthesis fails the request; you don't get a text-only answer in its place. A non-streaming request returns the 502 above. A stream that has started can't change its 200 status, so it ends with one error event carrying the same message, with no final finish_reason and no data: [DONE]. Treat that stream as failed: the audio you received stops partway through the answer. See Streaming errors.

Tool calling

Tool calling works alongside TTS. When the LLM decides to call a tool, the tool call is returned as standard delta.tool_calls chunks (no audio is generated for that turn). Once you execute the tool and send the result back with TTS enabled, the final response is spoken.

Tools and voice example

python
import requests
import json

API_URL = "https://api.inworld.ai/v1/chat/completions"
HEADERS = {
    "Authorization": "Basic <your-api-key>",
    "Content-Type": "application/json",
}

# Step 1: Request with tools + TTS enabled
response = requests.post(API_URL, headers=HEADERS, json={
    "model": "openai/gpt-5.4",
    "messages": [{"role": "user", "content": "What's the weather in Tokyo?"}],
    "tools": [{
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string"}
                },
                "required": ["location"]
            }
        }
    }],
    "audio": {
        "voice": "Dennis",
        "model": "inworld-tts-2"
    }
}).json()
# LLM returns tool_calls (no audio on this turn)

tool_call = response["choices"][0]["message"]["tool_calls"][0]

# Step 2: Execute the tool call
tool_result = get_weather("Tokyo")  # Your function

# Step 3: Send tool result back — this response is spoken aloud
audio_response = requests.post(API_URL, headers=HEADERS, json={
    "model": "openai/gpt-5.4",
    "stream": True,
    "messages": [
        {"role": "user", "content": "What's the weather in Tokyo?"},
        response["choices"][0]["message"],
        {"role": "tool", "content": tool_result, "tool_call_id": tool_call["id"]}
    ],
    "audio": {
        "voice": "Dennis",
        "model": "inworld-tts-2"
    }
}, stream=True)
# Parse SSE stream for audio chunks (same as Python example below)

Python Example

python
import requests
import json
import base64

response = requests.post(
    "https://api.inworld.ai/v1/chat/completions",
    headers={
        "Authorization": "Basic <your-api-key>",
        "Content-Type": "application/json",
    },
    json={
        "model": "openai/gpt-5.4",
        "max_tokens": 500,
        "stream": True,
        "audio": {
            "voice": "Dennis",
            "model": "inworld-tts-2",
        },
        "messages": [
            {"role": "user", "content": "Tell me a fun fact about space."}
        ],
    },
    stream=True,
)

response.raise_for_status()  # a 400 or 502 before the stream starts

audio_chunks: list[bytes] = []
full_transcript = ""
finished = False

for raw in response.iter_lines():
    line = raw.decode("utf-8")
    if not line.startswith("data: "):
        continue
    data = line[6:]
    if data == "[DONE]":
        finished = True
        break

    chunk = json.loads(data)
    if "error" in chunk:
        # Synthesis or the model failed after the stream started.
        raise RuntimeError(chunk["error"].get("message", "stream failed"))
    if not chunk.get("choices"):
        continue
    delta = chunk["choices"][0].get("delta", {})
    audio = delta.get("audio")

    if audio:
        # Text transcript — use for real-time text display
        if "transcript" in audio:
            full_transcript += audio["transcript"]
            print(audio["transcript"], end="", flush=True)

        # Audio data — PCM16 mono, 48 kHz by default
        if "data" in audio:
            audio_chunks.append(base64.b64decode(audio["data"]))

if not finished:
    raise RuntimeError("stream ended without [DONE]; the audio is incomplete")
pcm_audio = b"".join(audio_chunks)

JavaScript / Node.js Example

typescript
const response = await fetch("https://api.inworld.ai/v1/chat/completions", {
  method: "POST",
  headers: {
    Authorization: "Basic <your-api-key>",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "openai/gpt-5.4",
    max_tokens: 500,
    stream: true,
    audio: {
      voice: "Dennis",
      model: "inworld-tts-2",
    },
    messages: [
      { role: "user", content: "Tell me a fun fact about space." },
    ],
  }),
});

if (!response.ok || !response.body) {
  // A 400 or 502 before the stream starts.
  throw new Error(`HTTP ${response.status}: ${await response.text()}`);
}

const reader = response.body.getReader();
const decoder = new TextDecoder();
const audioChunks: Buffer[] = [];
let fullTranscript = "";
let buffer = "";
let finished = false;

while (!finished) {
  const { done, value } = await reader.read();
  if (done) break;

  buffer += decoder.decode(value, { stream: true });
  const lines = buffer.split("\n");
  buffer = lines.pop() ?? "";

  for (const line of lines) {
    if (!line.startsWith("data: ")) continue;
    const data = line.slice(6);
    if (data === "[DONE]") {
      finished = true;
      break;
    }

    const chunk = JSON.parse(data);
    if (chunk.error) {
      // Synthesis or the model failed after the stream started.
      throw new Error(chunk.error.message ?? "stream failed");
    }
    const audio = chunk.choices?.[0]?.delta?.audio;

    if (audio) {
      // Text transcript — use for real-time text display
      if (audio.transcript) {
        fullTranscript += audio.transcript;
        process.stdout.write(audio.transcript);
      }

      // Audio data — PCM16 mono, 48 kHz by default
      if (audio.data) {
        audioChunks.push(Buffer.from(audio.data, "base64"));
      }
    }
  }
}

if (!finished) {
  throw new Error("stream ended without [DONE]; the audio is incomplete");
}
const pcmAudio = Buffer.concat(audioChunks);

Optional timestamps

For captions or word highlighting, add "timestamp_type": "word" (or "character", lowercase) to your existing audio object. Omit it for audio without alignment; requesting timestamps can add latency.

Timestamp behavior differs between model generations. We recommend inworld-tts-2 or inworld-tts-2-flash for this feature. See the TTS timestamp documentation for model support and alignment details.

Read choices[].message.audio.timestamps for non-streaming responses, or collect choices[].delta.audio.timestamps when streaming. Entries look like:

json
{"token": "Hello", "start_time": 0.0, "end_time": 0.35}

Times are seconds from the beginning of the complete response audio, including preceding sentences. Streaming timing entries may arrive separately from audio, without data or transcript; collect them independently and do not assume they arrive before playback.

Next Steps