Routing

Request-level routing

Route requests without creating a router

With the Chat Completions API, you can route requests without creating a router. In a single request, you can call a specific model, choose its provider, add fallbacks, or let the router choose a model for you.

Use request-level routing to:

Call a specific model through a unified API without setting up a router
Prototype or benchmark before choosing a router configuration

For conditional routing, A/B testing with weighted variants, or shared prompt templates, set up a router.

Authentication

Run these examples on your server. Set INWORLD_API_KEY to the Base64 credentials copied from Portal or the CLI. Use the value as-is; don't Base64-encode it again. A Standard key can call chat completions. You only need Router Write permission to change router configuration.

The direct HTTP examples use Basic. Keep API keys on your server. Browser and mobile clients should use a token created by your backend or a server proxy.

Direct model call

Set model to a provider/model identifier:

bash

curl -X POST https://api.inworld.ai/v1/chat/completions \
  -H "Authorization: Basic $INWORLD_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-5.2",
    "messages": [{ "role": "user", "content": "Hello!" }]
  }'

This calls the specified model through Inworld Inference's unified API, with no routing logic. A provider/model identifier selects that provider. The router never switches it to another provider's copy of the model, and extra_body.provider and extra_body.ignore don't apply to it. To fall back to other models or providers, list them in extra_body.models.

Provider selection

Some models are offered by more than one provider. Set model to the model name without a provider prefix (for example, deepseek-v4-flash) to let the router pick a provider. By default, it tries the lowest-latency provider first and moves to the next provider if that one fails.

Use extra_body.provider to choose which providers are tried and in what order. No router is needed:

bash

curl -X POST https://api.inworld.ai/v1/chat/completions \
  -H "Authorization: Basic $INWORLD_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "deepseek-v4-flash",
    "messages": [{ "role": "user", "content": "Hello!" }],
    "extra_body": {
      "provider": {
        "order": ["deepseek", "inworld"],
        "allow_fallbacks": true
      }
    }
  }'

This tries deepseek/deepseek-v4-flash, then inworld/deepseek-v4-flash. Providers not in order are not used. Set allow_fallbacks to false to try only the first provider. provider applies only when model has no provider prefix. It is ignored when model is a provider/model identifier or auto. See Provider routing for all options and the full execution order.

Which model served the request

The response's model field is the provider/model identifier that produced the answer. Its prefix is the provider. Each entry in metadata.attempts has the provider/model it tried, success, and, for a failed attempt, status_code and error. See the Attempt schema for every field.

Fallbacks

Add fallback models via extra_body.models. If the primary model fails, the router automatically tries the next model in the list:

bash

curl -X POST https://api.inworld.ai/v1/chat/completions \
  -H "Authorization: Basic $INWORLD_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-5.2",
    "messages": [{ "role": "user", "content": "Hello!" }],
    "extra_body": {
      "models": ["anthropic/claude-opus-4-6", "google-ai-studio/gemini-2.5-pro"]
    }
  }'

In this example, the router tries gpt-5.2 first, then Claude Opus, then Gemini Pro. Fallback models are tried in the order listed. If you also set extra_body.sort, the router sorts the fallback models by those criteria instead. The response's metadata.attempts array shows which models it tried.

Router fallbacks run within one HTTP request. SDK retries resend the original request, so each retry can run the whole fallback list again.

The OpenAI and Anthropic Python and TypeScript SDKs retry connection errors, timeouts and some HTTP errors twice by default, for up to three HTTP attempts. Set max_retries (Python) or maxRetries (TypeScript and JavaScript) together with the SDK's timeout. The timeout applies to each attempt, so it can end a request before the router has tried every fallback, and the retry then starts the list again. For a strict limit on total latency, enforce one overall deadline that covers every attempt and the delays between retries. The OpenAI and Anthropic SDK docs list which errors they retry.

Once a stream has started, neither fallbacks nor SDK retries apply. Handle errors after the first chunk as described in Streaming.

Fallback by first token timeout

Set a time to first token (TTFT) timeout to trigger fallback when a model is slow. If the model doesn't return its first token in time, the router cancels the request and tries the next model.

Use this when your application needs a fast response and can try another model instead of waiting.

Set extra_body.fallback.ttft_timeout in your request:

bash

curl -X POST https://api.inworld.ai/v1/chat/completions \
  -H "Authorization: Basic $INWORLD_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-5.2",
    "messages": [{ "role": "user", "content": "Hello" }],
    "extra_body": {
      "models": ["openai/gpt-4o", "google-ai-studio/gemini-2.5-pro"],
      "fallback": {
        "ttft_timeout": "900ms"
      }
    }
  }'

The ttft_timeout value is a duration string, such as "300ms", "1s", or "1.5s". The minimum is 300ms.

The timeout applies to every model in the list, including the last one. If no model sends its first token in time, the request fails with a 503 error whose message lists each model as Timed out waiting for first token. The OpenAI and Anthropic SDKs retry a 503 by default, so each of their retries runs the whole list again, as described in the note above. A model that reasons before it answers can take longer to send its first token, so set the timeout with the request's reasoning settings in mind.

Auto model selection

Set model to auto and use extra_body.sort to tell the router how to rank models:

bash

curl -X POST https://api.inworld.ai/v1/chat/completions \
  -H "Authorization: Basic $INWORLD_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "auto",
    "messages": [{ "role": "user", "content": "Hello!" }],
    "extra_body": {
      "sort": ["price"]
    }
  }'

This selects the cheapest available model. Available sort criteria: price, latency, throughput, intelligence, math, coding. Add _asc or _desc to set the direction, for example latency_desc.

In a request, extra_body.sort is an array of strings, such as ["price"]. In a saved router, model_selection.sort is an array of objects, such as [{"metric": "SORT_METRIC_PRICE"}]. Each form fails in the other place.

You can combine criteria. The router ranks models by the first criterion and uses the others to break ties:

bash

curl -X POST https://api.inworld.ai/v1/chat/completions \
  -H "Authorization: Basic $INWORLD_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "auto",
    "messages": [{ "role": "user", "content": "Hello!" }],
    "extra_body": {
      "sort": ["price", "latency"]
    }
  }'

This picks the cheapest model, using latency as a tiebreaker.

Filtering models

Use extra_body.models to limit which models the router can choose. Use extra_body.ignore to exclude models or entire providers:

Restrict to specific models

{
  "model": "auto",
  "messages": [{ "role": "user", "content": "Hello!" }],
  "extra_body": {
    "models": ["openai/gpt-5.2", "anthropic/claude-opus-4-6", "google-ai-studio/gemini-2.5-pro"],
    "sort": ["latency"]
  }
}

Exclude models or providers

{
  "model": "auto",
  "messages": [{ "role": "user", "content": "Hello!" }],
  "extra_body": {
    "ignore": ["openai", "anthropic/claude-opus-4-6"],
    "sort": ["latency"]
  }
}