Telephony (Twilio)

Twilio ConversationRelay handles call audio and speech-to-text, then streams user transcripts to your server over a WebSocket. Your server calls LLM Router with voice responses (LLM + TTS) to receive the LLM response and synthesized audio in one streaming request. Your server sends the audio back to ConversationRelay for playback.

Prerequisites

• Node.js v20 or later
• ngrok account with a reserved static domain (the free tier is sufficient)
• Twilio account with a phone number that has Voice capability.
• Inworld account with a Router API key

Setup

The steps below walk through the reference implementation in inworld-ai/inworld-api-examples.

1. Clone the example repo

git clone https://github.com/inworld-ai/inworld-api-examples.git
cd inworld-api-examples/integrations/twilio-conversation-relay

The remaining steps run from this directory.

2. Get your Inworld API key

Sign in to the Inworld Portal, open your workspace, and create an API key. The same key works for Router (LLM) and TTS because the integration uses the combined LLM + TTS endpoint.

3. Get a Twilio phone number

In the Twilio Console, buy a phone number with Voice capability. This is the number callers will dial.

4. Reserve an ngrok static domain (for local development)

Install ngrok and reserve a free static domain in the ngrok dashboard. A static domain matters here because Twilio's webhook URL needs to stay stable between restarts. Without one, every new ngrok session changes the tunnel URL and you have to update the Twilio webhook by hand.

5. Configure environment

Copy the example env file and fill in the required variables:

cp .env.example .env

At minimum, set:

INWORLD_API_KEY=your_inworld_api_key
SERVER_URL=https://your-ngrok-domain.ngrok-free.app

Optional variables (with sensible defaults) let you change the system prompt, LLM model, TTS voice and model, STT provider, and welcome greeting. See the repo README for the full table.

6. Install and run

Install dependencies:

npm install

Then start the dev server and ngrok in two separate terminals:

npm run dev

ngrok http 3000 --url=your-ngrok-domain.ngrok-free.app

7. Point your Twilio number at the webhook

In the Twilio Console, go to Phone Numbers → Manage → Active Numbers → your number. Under Voice Configuration, set A call comes in to Webhook, enter https://your-ngrok-domain.ngrok-free.app/voice, and choose HTTP POST. Save the configuration.

How it works

• An inbound call hits /voice, and the server returns TwiML that hands the call off to ConversationRelay.
• ConversationRelay handles the call audio and runs speech-to-text, then opens a WebSocket to your server and streams user transcripts as they arrive.
• For each user turn, the server calls the LLM Router API's chat completions endpoint with an audio block. The server receives an SSE stream of base64-encoded PCM audio chunks, with each segment's text arriving alongside its audio as a transcript. This is the LLM + TTS feature.
• Inworld's chunking engine groups the response into text segments at natural sentence boundaries, and each segment's audio may arrive as multiple base64-encoded PCM chunks. The server assembles each segment's audio, wraps it in a WAV header, and hosts it at a short-lived HTTP URL. The server then sends a play message to ConversationRelay. The caller hears the first sentence before the LLM finishes generating the rest.
• On an interruption (barge-in), ConversationRelay sends an interrupt message and the server aborts the in-flight Inworld stream via AbortController. Partial responses already spoken are kept in conversation history so context is preserved.

The TwiML returned from /voice looks like this:

<Response>
  <Connect>
    <ConversationRelay
      url="wss://your-ngrok-domain.ngrok-free.app/conversation"
      interruptible="true"
      dtmfDetection="true"
    />
  </Connect>
</Response>

Test your integration

Call your Twilio number. The bot should greet you and hold a conversation.

Example implementation

Further reading