Voice responses (LLM + TTS)

Generate a model answer and synthesize it with Inworld TTS in one chat completions request.

Overview

Voice responses generate an answer and speak it in one API call (LLM + TTS). Add audio to a chat completion: Inworld Inference generates the answer with your chosen model, and Inworld TTS speaks it. You receive both text and audio, with or without streaming.

Voice responses take the conversation as messages and speak the answer. For a live conversation with speech input, turn detection, and interruptions, use the Realtime API. To synthesize text you already have, use TTS.

In streaming mode, text from the selected model is sent to Inworld TTS as it is generated, and the response contains audio chunks with their transcript. In non-streaming mode, the complete audio and transcript are returned together once the response is ready.

This is ideal for:

Voice assistants and conversational agents
Real-time narration and read-aloud features
Accessibility-first applications
Any workflow where your users hear AI responses instead of (or in addition to) reading them

Choose a model

Choose a text model from inworld.ai/models and copy its API model ID. The example uses openai/gpt-5.4 for the answer and inworld-tts-2 for speech.

Generate a spoken response

Add the audio parameter to your chat completions request. Run the request on your server with the complete Base64 credential from Portal in INWORLD_API_KEY; do not encode it again. See the Inference quickstart for setup.

bash

curl --request POST \
  --url https://api.inworld.ai/v1/chat/completions \
  --header "Authorization: Basic $INWORLD_API_KEY" \
  --header 'Content-Type: application/json' \
  --data '{
    "model": "openai/gpt-5.4",
    "max_tokens": 1000,
    "stream": true,
    "audio": {
      "voice": "Dennis",
      "model": "inworld-tts-2"
    },
    "messages": [
      {"role": "user", "content": "What is the meaning of life?"}
    ]
  }'

That's it. Inworld Inference will:

Generate an answer with the selected model
Stream text chunks to Inworld TTS as they're generated
Return both text and audio in the SSE stream

Audio Parameters

The audio object controls voice synthesis:

Parameter	Type	Description
`voice`	string	Required. The voice ID to use for speech synthesis (e.g., `"Dennis"`, `"Chloe"`). See List Voices for all available voices.
`model`	string	Required. The TTS model to use (e.g., `"inworld-tts-2"`). See TTS Models for available options.
`format`	string	Optional: `"pcm16"` (default), `"mulaw"` or `"alaw"`. See Audio formats.
`sample_rate`	integer	Optional: the output sample rate in Hz. See Audio formats for the rates each format accepts.
`timestamp_type`	string	Optional: `"word"` or `"character"`. See Optional timestamps.

Audio formats

Audio is always mono and headerless, in both streaming and non-streaming responses.

`format`	Encoding	`sample_rate` values	Default rate
`pcm16`	16-bit signed little-endian PCM	8000, 16000, 22050, 24000, 32000, 44100, 48000	48000
`mulaw`	G.711 µ-law, 8-bit	8000	8000
`alaw`	G.711 A-law, 8-bit	8000	8000

Any other format, including OpenAI's wav, mp3, flac, opus and aac, or a sample_rate the format does not accept, returns 400. Use mulaw or alaw for telephony that expects G.711 audio.

Streaming Response Format

When streaming is enabled ("stream": true), the response is delivered as Server-Sent Events (SSE). Each event is a JSON object in the data field.

When TTS is active, text is delivered through delta.audio.transcript. Audio data and its corresponding transcript are sent together via delta.audio:

json

data: {"choices":[{"delta":{"audio":{"data":"<base64-audio>","transcript":"Hello! How can I assist you today?"}},"index":0}],...}

Field	Description
`delta.audio.data`	Base64-encoded audio in the requested format.
`delta.audio.transcript`	The text being spoken. Use this for real-time text display.

Text and audio are chunked independently. Text is chunked at natural sentence boundaries, while audio arrives in frames. This means a single transcript value may span multiple audio chunks. The transcript for a text segment is attached to the first audio chunk of that segment — subsequent audio chunks for the same segment omit transcript.

Non-Streaming Response

Without streaming ("stream": false), the full audio and transcript are returned in the message.audio object:

json

{
  "choices": [{
    "message": {
      "role": "assistant",
      "content": "",
      "audio": {
        "id": "audio_chatcmpl-xyz",
        "data": "<base64-audio>",
        "transcript": "Hello! How can I assist you today?"
      }
    },
    "finish_reason": "stop"
  }]
}

When TTS is active, message.content is empty. The full text is available in message.audio.transcript.

Choose another text model

Choose the text model for the answer separately from the TTS model and voice. Browse inworld.ai/models for model IDs. You can also use optional automatic model selection with voice responses:

bash

# Use auto model selection + TTS
curl --request POST \
  --url https://api.inworld.ai/v1/chat/completions \
  --header "Authorization: Basic $INWORLD_API_KEY" \
  --header 'Content-Type: application/json' \
  --data '{
    "model": "auto",
    "stream": true,
    "audio": {
      "voice": "Chloe",
      "model": "inworld-tts-2"
    },
    "messages": [
      {"role": "user", "content": "Tell me a short bedtime story."}
    ],
    "extra_body": {
      "sort": ["latency"]
    }
  }'

This example requests automatic model selection with latency ordering, then synthesizes the answer with Inworld TTS. Latency ordering does not guarantee that a particular request will be served by the fastest model.

Add routing when needed

You can add fallbacks or model-selection preferences to a voice-response request:

Fallbacks with voice

Configure backup models for failures before the response starts. See Fallbacks for retry behavior and streaming limits:

bash

curl --request POST \
  --url https://api.inworld.ai/v1/chat/completions \
  --header "Authorization: Basic $INWORLD_API_KEY" \
  --header 'Content-Type: application/json' \
  --data '{
    "model": "openai/gpt-5.4",
    "stream": true,
    "audio": {
      "voice": "Dennis",
      "model": "inworld-tts-2"
    },
    "messages": [
      {"role": "user", "content": "Explain quantum computing simply."}
    ],
    "extra_body": {
      "models": ["anthropic/claude-opus-4-6", "google-ai-studio/gemini-2.5-flash"]
    }
  }'

If GPT-5 fails, the request falls over to Claude or Gemini — and the same voice (Dennis) is used regardless of which model generates the text.

Cost-Optimized Voice Responses

Route to the cheapest model while still getting audio output:

bash

curl --request POST \
  --url https://api.inworld.ai/v1/chat/completions \
  --header "Authorization: Basic $INWORLD_API_KEY" \
  --header 'Content-Type: application/json' \
  --data '{
    "model": "auto",
    "stream": true,
    "audio": {
      "voice": "Dennis",
      "model": "inworld-tts-2"
    },
    "messages": [
      {"role": "user", "content": "What is the capital of France?"}
    ],
    "extra_body": {
      "sort": ["price", "latency"]
    }
  }'

Stream text into speech

Inworld Inference splits generated text into chunks for TTS. Before the LLM finishes the full response, the router:

Buffers incoming tokens from the LLM
Detects sentence boundaries: sentence-ending punctuation (., !, ? and their CJK equivalents) and line breaks
Sends each sentence to the TTS engine as soon as it's complete

This allows speech synthesis to start before the full text response is complete. Commas, colons and semicolons never split a chunk, so each sentence is synthesized with its full context, producing smooth, conversational audio.

Tool calling

Tool calling works alongside TTS. When the LLM decides to call a tool, the tool call is returned as standard delta.tool_calls chunks (no audio is generated for that turn). Once you execute the tool and send the result back with TTS enabled, the final response is spoken.

Tools and voice example

python

import requests
import json

API_URL = "https://api.inworld.ai/v1/chat/completions"
HEADERS = {
    "Authorization": "Basic <your-api-key>",
    "Content-Type": "application/json",
}

# Step 1: Request with tools + TTS enabled
response = requests.post(API_URL, headers=HEADERS, json={
    "model": "openai/gpt-5.4",
    "messages": [{"role": "user", "content": "What's the weather in Tokyo?"}],
    "tools": [{
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string"}
                },
                "required": ["location"]
            }
        }
    }],
    "audio": {
        "voice": "Dennis",
        "model": "inworld-tts-2"
    }
}).json()
# LLM returns tool_calls (no audio on this turn)

tool_call = response["choices"][0]["message"]["tool_calls"][0]

# Step 2: Execute the tool call
tool_result = get_weather("Tokyo")  # Your function

# Step 3: Send tool result back — this response is spoken aloud
audio_response = requests.post(API_URL, headers=HEADERS, json={
    "model": "openai/gpt-5.4",
    "stream": True,
    "messages": [
        {"role": "user", "content": "What's the weather in Tokyo?"},
        response["choices"][0]["message"],
        {"role": "tool", "content": tool_result, "tool_call_id": tool_call["id"]}
    ],
    "audio": {
        "voice": "Dennis",
        "model": "inworld-tts-2"
    }
}, stream=True)
# Parse SSE stream for audio chunks (same as Python example below)

Python Example

python

import requests
import json
import base64

response = requests.post(
    "https://api.inworld.ai/v1/chat/completions",
    headers={
        "Authorization": "Basic <your-api-key>",
        "Content-Type": "application/json",
    },
    json={
        "model": "openai/gpt-5.4",
        "max_tokens": 500,
        "stream": True,
        "audio": {
            "voice": "Dennis",
            "model": "inworld-tts-2",
        },
        "messages": [
            {"role": "user", "content": "Tell me a fun fact about space."}
        ],
    },
    stream=True,
)

audio_chunks = []
full_transcript = ""

for line in response.iter_lines():
    line = line.decode("utf-8")
    if not line.startswith("data: "):
        continue
    data = line[6:]
    if data == "[DONE]":
        break

    chunk = json.loads(data)
    delta = chunk["choices"][0].get("delta", {})
    audio = delta.get("audio")

    if audio:
        # Text transcript — use for real-time text display
        if "transcript" in audio:
            full_transcript += audio["transcript"]
            print(audio["transcript"], end="", flush=True)

        # Audio data — PCM16 mono, 48 kHz by default
        if "data" in audio:
            audio_chunks.append(base64.b64decode(audio["data"]))

pcm_audio = b"".join(audio_chunks)

JavaScript / Node.js Example

javascript

const response = await fetch("https://api.inworld.ai/v1/chat/completions", {
  method: "POST",
  headers: {
    Authorization: "Basic <your-api-key>",
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    model: "openai/gpt-5.4",
    max_tokens: 500,
    stream: true,
    audio: {
      voice: "Dennis",
      model: "inworld-tts-2",
    },
    messages: [
      { role: "user", content: "Tell me a fun fact about space." },
    ],
  }),
});

const reader = response.body.getReader();
const decoder = new TextDecoder();
const audioChunks = [];
let fullTranscript = "";
let buffer = "";

while (true) {
  const { done, value } = await reader.read();
  if (done) break;

  buffer += decoder.decode(value, { stream: true });
  const lines = buffer.split("\n");
  buffer = lines.pop();

  for (const line of lines) {
    if (!line.startsWith("data: ")) continue;
    const data = line.slice(6);
    if (data === "[DONE]") break;

    const chunk = JSON.parse(data);
    const audio = chunk.choices[0]?.delta?.audio;

    if (audio) {
      // Text transcript — use for real-time text display
      if (audio.transcript) {
        fullTranscript += audio.transcript;
        process.stdout.write(audio.transcript);
      }

      // Audio data — PCM16 mono, 48 kHz by default
      if (audio.data) {
        audioChunks.push(Buffer.from(audio.data, "base64"));
      }
    }
  }
}

const pcmAudio = Buffer.concat(audioChunks);

Optional timestamps

For captions or word highlighting, add "timestamp_type": "word" (or "character", lowercase) to your existing audio object. Omit it for audio without alignment; requesting timestamps can add latency.

Timestamp behavior differs between model generations. We recommend inworld-tts-2 or inworld-tts-2-flash for this feature. See the TTS timestamp documentation for model support and alignment details.

Read choices[].message.audio.timestamps for non-streaming responses, or collect choices[].delta.audio.timestamps when streaming. Entries look like:

json

{"token": "Hello", "start_time": 0.0, "end_time": 0.35}

Times are seconds from the beginning of the complete response audio, including preceding sentences. Streaming timing entries may arrive separately from audio, without data or transcript; collect them independently and do not assume they arrive before playback.

Next Steps

OpenAI Compatibility — use Inworld Inference with the OpenAI SDK
Cost Optimizer — route by query complexity to reduce costs
Failover System — build a resilient multi-provider setup
Extra Body Parameters — all available sort, models, and ignore options