Twilio Integration

Twilio Media Streams forward live call audio to your server over a WebSocket. Because the Realtime API natively accepts G.711 μ-law (audio/pcmu) at 8 kHz, you can pipe Twilio audio straight through without transcoding. A single Realtime connection handles STT, LLM, and TTS, so the bridge server is mostly glue.

By default the example runs all three stages on Inworld: Inworld STT (inworld/inworld-stt-1), DeepSeek V4.1 Flash hosted by Inworld (inworld/models/deepseek-v4.1-flash), and Inworld TTS (inworld-tts-2). The caller's language is detected automatically, so callers can speak languages other than English.

Prerequisites

• Node.js v18 or later
• ngrok account (the free tier is sufficient and includes one static dev domain)
• Twilio account with a phone number that has Voice capability
• Inworld account

Setup

The steps below walk through the reference implementation in inworld-ai/inworld-api-examples.

1. Clone the example repo

Clone the examples repo and change into the Twilio integration directory:

git clone https://github.com/inworld-ai/inworld-api-examples.git
cd inworld-api-examples/integrations/twilio

The remaining steps are run from this directory.

2. Get your Inworld API key

Sign in to the Inworld Portal, open your workspace, and create an API key. For Type, choose Realtime-only: this example only calls the Realtime API, so a scoped key is enough. Copy the Base64 credential. The server sends it as Authorization: Basic <key>.

3. Get a Twilio phone number

In the Twilio Console, buy a phone number with Voice capability. This is the number callers will dial.

4. Set up ngrok

Install ngrok (for example, brew install ngrok on macOS), then connect it to your account with your authtoken:

ngrok config add-authtoken <your-authtoken>

Every free ngrok account includes one static dev domain, listed under Domains in the ngrok dashboard (for example, your-name.ngrok-free.dev). You don't need to reserve it. A static domain matters here because Twilio's webhook URL needs to stay stable between restarts. Without one, every new ngrok session changes the tunnel URL and you have to update the Twilio webhook by hand.

5. Configure environment

Copy the example env file and fill in the two required variables:

cp .env.example .env

Set these values in .env:

INWORLD_API_KEY=your_base64_inworld_api_key
SERVER_URL=https://your-ngrok-domain.ngrok-free.dev

SERVER_URL is the bare domain, without a trailing /voice.

Optional variables:

6. (Optional) Choose a different LLM

The default, DeepSeek V4.1 Flash hosted by Inworld, is fast for voice: in testing it started speaking about 0.35 s sooner than openai/gpt-4.1-mini. To use another model, set LLM_MODEL in .env to any model from the Inworld models list:

# Another Inworld-hosted model
LLM_MODEL=inworld/models/<model>
# A third-party model
LLM_MODEL=openai/gpt-4.1-mini

For automatic fallbacks or A/B tests between models, route the LLM through an Inworld Router instead:

1. In the Inworld Portal, create a router and give it an ID, for example twilio-realtime. The ID can't be changed later.

2. Add a default route with one variant, for example inworld/models/deepseek-v4.1-flash with openai/gpt-4.1-nano as a fallback model.

3. Leave the prompt template and advanced model settings empty. The system prompt comes from SYSTEM_PROMPT, and reasoning adds latency before the first word.

4. Set the router in .env:

   LLM_MODEL=inworld/twilio-realtime

A router adds about 0.2 s per reply compared with calling the same model directly, so use one when you need its features.

7. Install and run

Install dependencies:

npm install

Then start ngrok and the dev server in two separate terminals:

ngrok http 3000 --url=your-ngrok-domain.ngrok-free.dev

npm run dev

8. Point your Twilio number at the webhook

1. In the Twilio Console, go to Phone Numbers → Manage → Active numbers and select your number.
2. Under Configuration details → Voice and emergency address, click Edit configuration details.
3. In Handling for incoming calls, set the primary method to Webhook, the URL to https://your-ngrok-domain.ngrok-free.dev/voice, and the HTTP format to POST.
4. Save. The backup webhook and call status change URLs can stay empty.

How it works

• An inbound call hits /voice, and the server responds with TwiML instructing Twilio to open a Media Stream.
• Twilio opens a WebSocket to /media-stream and begins forwarding call audio.
• The server shuttles mulaw 8 kHz frames between Twilio and Inworld in both directions. No format conversion is required.
• Inworld transcribes the caller with Inworld STT, detects the end of each turn with semantic VAD (eagerness: "medium"), and generates a spoken reply.
• On detected caller speech, the server clears Twilio's audio buffer so playback stops right away. Inworld cancels the in-flight response itself because the session sets interrupt_response: true, so the server doesn't send response.cancel.

The TwiML returned from /voice looks like this:

<Response>
  <Connect>
    <Stream url="wss://your-ngrok-domain.ngrok-free.dev/media-stream"/>
  </Connect>
</Response>

Test your integration

Call your Twilio number. The bot should greet you and hold a conversation. Try interrupting it mid-answer; it should stop and respond to what you said.

The server logs every turn, so you can check a call without guessing:

[latency] Turn end detected: +1314ms after caller stopped talking
[call] User: What's the weather in Seattle?
[latency] First bot audio: +2378ms after caller stopped talking
[call] Bot (completed, 10760ms audio, response <id>): ...
[call] Caller interrupted the bot, clearing ~7196ms of unplayed audio

• Turn end detected is how long turn detection waited after the caller stopped talking. The gap between it and First bot audio is LLM plus TTS time. Neither includes phone network delay.
• Bot (cancelled, ...) means the reply was interrupted, usually because the caller kept talking after a pause mid-sentence.

Example implementation

Further reading