Realtime TTS-2 is live. Built for realtime conversation that feels human. Read the Realtime TTS-2 announcement

Routing

Request-level routing

Route requests without creating a router

With the Chat Completions API, you can route requests without creating a router. In a single request, you can call a specific model, choose its provider, add fallbacks, or let the router choose a model for you.

Use request-level routing to:

  • Call a specific model through a unified API without setting up a router
  • Prototype or benchmark before choosing a router configuration

For conditional routing, A/B testing with weighted variants, or shared prompt templates, set up a router.

Authentication

Run these examples on your server. Set INWORLD_API_KEY to the Base64 credentials copied from Portal or the CLI. Use the value as-is; don't Base64-encode it again. A Standard key can call chat completions. You only need Router Write permission to change router configuration.

The direct HTTP examples use Basic. Keep API keys on your server. Browser and mobile clients should use a token created by your backend or a server proxy.

Direct model call

Set model to a provider/model identifier:

bash
curl -X POST https://api.inworld.ai/v1/chat/completions \
  -H "Authorization: Basic $INWORLD_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-5.2",
    "messages": [{ "role": "user", "content": "Hello!" }]
  }'

This calls the specified model through Inworld Inference's unified API, with no routing logic. A provider/model identifier selects that provider. The router never switches it to another provider's copy of the model, extra_body.provider doesn't apply to it, and extra_body.ignore never removes it. To fall back to other models or providers, list them in extra_body.models.

Model ids

Set model to one of these forms:

FormExampleWhat the router does
provider/modelopenai/gpt-5.2, groq/openai/gpt-oss-120bCalls that provider's copy of the model, and no other provider's.
Model name alonedeepseek-v4-flashPicks a provider that serves the model. See Provider selection.
inworld/models/<name>inworld/models/deepseek-v4-flashCalls a model that Inworld hosts.
inworld/<router-id>inworld/my-routerSends the request through your saved router.
autoautoPicks a model for you. See Auto model selection.

Take the id from the List models response: each entry's id is the value to send as model, such as openai/gpt-5.2 or inworld/models/deepseek-v4-flash. Send it unchanged. It is the entry's provider and model joined by /, which matters for the two cases below.

  • Model names that contain a slash. Some model names include their creator, such as openai/gpt-oss-120b, which Groq and DeepInfra serve. Their ids are groq/openai/gpt-oss-120b and deepinfra/openai/gpt-oss-120b. The router splits an id at its first /. When the part before it is a provider in the catalog, that part is the provider. So openai/gpt-oss-120b asks the provider openai for a model named gpt-oss-120b. OpenAI doesn't serve it, so the request fails, and the error message lists the ids that serve the model. When the part before the first / isn't a provider (for example, Qwen in Qwen/Qwen3-235B-A22B-Instruct-2507), the router looks up the whole string as a model name and uses a provider that serves it. In both cases extra_body.provider doesn't apply, because the id contains a /. To fall back to another provider's copy, list its id in extra_body.models.
  • Models that Inworld hosts. List models returns them with provider inworld, model models/<name> and id inworld/models/<name>. Don't set model to inworld/<name>: that form names a saved router, and the request fails with 404 if you have no router with that name. The response's model and metadata.attempts show these models without models/, as inworld/<name>. Use that form in extra_body.models, extra_body.ignore, and a router variant's model_id.

Provider selection

Some models are offered by more than one provider. Set model to the model name without a provider prefix (for example, deepseek-v4-flash) to let the router pick a provider. By default, it tries the lowest-latency provider first and moves to the next provider if that one fails.

Use extra_body.provider to choose which providers are tried and in what order. No router is needed:

bash
curl -X POST https://api.inworld.ai/v1/chat/completions \
  -H "Authorization: Basic $INWORLD_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "deepseek-v4-flash",
    "messages": [{ "role": "user", "content": "Hello!" }],
    "extra_body": {
      "provider": {
        "order": ["deepseek", "inworld"],
        "allow_fallbacks": true
      }
    }
  }'

This tries deepseek/deepseek-v4-flash, then inworld/deepseek-v4-flash. Providers not in order are not used. Set allow_fallbacks to false to try only the first provider. extra_body.models and extra_body.ignore can still add fallbacks on other providers; to keep every attempt on one provider, see Use only one provider. provider applies only when model has no provider prefix. It is ignored when model is a provider/model identifier or auto. See Provider routing for all options and the full execution order.

Which model served the request

The response's model field is the provider/model identifier that produced the answer. The part before its first / is the provider. A model that Inworld hosts shows as inworld/<name>; to request it again, set model to inworld/models/<name> (see Model ids). Each entry in metadata.attempts has the provider/model it tried, success, and, for a failed attempt, status_code and error. See the Attempt schema for every field.

Fallbacks

Add fallback models via extra_body.models. If the primary model fails, the router automatically tries the next model in the list:

bash
curl -X POST https://api.inworld.ai/v1/chat/completions \
  -H "Authorization: Basic $INWORLD_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-5.2",
    "messages": [{ "role": "user", "content": "Hello!" }],
    "extra_body": {
      "models": ["anthropic/claude-opus-4-6", "google-ai-studio/gemini-2.5-pro"]
    }
  }'

In this example, the router tries gpt-5.2 first, then Claude Opus, then Gemini Pro. Fallback models are tried in the order listed. If you also set extra_body.sort, the router sorts the fallback models by those criteria instead. The response's metadata.attempts array shows which models it tried.

Router fallbacks run within one HTTP request. SDK retries resend the original request, so each retry can run the whole fallback list again.

The OpenAI and Anthropic Python and TypeScript SDKs retry connection errors, timeouts and some HTTP errors twice by default, for up to three HTTP attempts. Set max_retries (Python) or maxRetries (TypeScript and JavaScript) together with the SDK's timeout. The timeout applies to each attempt, so it can end a request before the router has tried every fallback, and the retry then starts the list again. For a strict limit on total latency, enforce one overall deadline that covers every attempt and the delays between retries. The OpenAI and Anthropic SDK docs list which errors they retry.

Once a stream has started, neither fallbacks nor SDK retries apply. Handle errors after the first chunk as described in Streaming.

Fallback by first token timeout

Set a time to first token (TTFT) timeout to trigger fallback when a model is slow. If the model doesn't return its first token in time, the router cancels the request and tries the next model.

Use this when your application needs a fast response and can try another model instead of waiting.

Set extra_body.fallback.ttft_timeout in your request:

bash
curl -X POST https://api.inworld.ai/v1/chat/completions \
  -H "Authorization: Basic $INWORLD_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-5.2",
    "messages": [{ "role": "user", "content": "Hello" }],
    "extra_body": {
      "models": ["openai/gpt-4o", "google-ai-studio/gemini-2.5-pro"],
      "fallback": {
        "ttft_timeout": "900ms"
      }
    }
  }'

The ttft_timeout value is a duration string, such as "300ms", "1s", or "1.5s". The minimum is 300ms.

The timeout applies to every model in the list, including the last one. If no model sends its first token in time, the request fails with a 503 error whose message lists each model as Timed out waiting for first token. The OpenAI and Anthropic SDKs retry a 503 by default, so each of their retries runs the whole list again, as described in the note above. A model that reasons before it answers can take longer to send its first token, so set the timeout with the request's reasoning settings in mind.

Auto model selection

Set model to auto and use extra_body.sort to tell the router how to rank models:

bash
curl -X POST https://api.inworld.ai/v1/chat/completions \
  -H "Authorization: Basic $INWORLD_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "auto",
    "messages": [{ "role": "user", "content": "Hello!" }],
    "extra_body": {
      "sort": ["price"]
    }
  }'

This selects the cheapest available model. Available sort criteria: price, latency, throughput, intelligence, math, coding. Add _asc or _desc to set the direction, for example latency_desc.

In a request, extra_body.sort is an array of strings, such as ["price"]. In a saved router, model_selection.sort is an array of objects, such as [{"metric": "SORT_METRIC_PRICE"}]. Each form fails in the other place.

You can combine criteria. The router ranks models by the first criterion and uses the others to break ties:

bash
curl -X POST https://api.inworld.ai/v1/chat/completions \
  -H "Authorization: Basic $INWORLD_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "auto",
    "messages": [{ "role": "user", "content": "Hello!" }],
    "extra_body": {
      "sort": ["price", "latency"]
    }
  }'

This picks the cheapest model, using latency as a tiebreaker.

Filtering models

Use extra_body.models to limit which models the router can choose. Use extra_body.ignore to exclude models or entire providers:

Restrict to specific models
{
  "model": "auto",
  "messages": [{ "role": "user", "content": "Hello!" }],
  "extra_body": {
    "models": ["openai/gpt-5.2", "anthropic/claude-opus-4-6", "google-ai-studio/gemini-2.5-pro"],
    "sort": ["latency"]
  }
}