Realtime TTS-2 is live. Built for realtime conversation that feels human. Read the Realtime TTS-2 announcement

Routing

Provider routing

Route requests across providers for the same model

Specify a model without a provider prefix (e.g., deepseek-v4-flash instead of deepseek/deepseek-v4-flash) to let the API select a provider. A model with a provider prefix is never switched to another provider. For how the router reads an id, including model names that contain a /, see Model ids.

By default, the provider with the lowest latency is selected, and if it fails, the next best provider is tried automatically.

To control provider selection, set a provider object in either place. You don't need a router for it:

bash
curl -X POST https://api.inworld.ai/v1/chat/completions \
  -H "Authorization: Basic $INWORLD_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "deepseek-v4-flash",
    "messages": [{ "role": "user", "content": "Hello!" }],
    "extra_body": {
      "provider": {
        "order": ["deepseek", "inworld"]
      }
    }
  }'

provider applies only when the model has no provider prefix. It is ignored when the model is a provider/model identifier or auto.

Use only one provider

To send every attempt to one provider, use either of these, with no models and no ignore:

  • Name the provider in model, for example "model": "deepseek/deepseek-v4-flash".
  • Send the model name alone with provider.order set to that one provider, for example "provider": { "order": ["deepseek"] }.

models and ignore both add fallbacks that can reach other providers, even when allow_fallbacks is false:

  • models adds the models it lists, after the primary model.
  • ignore without models adds every other model in the catalog that ignore doesn't exclude, from any provider. This applies to any model except auto and a saved router.

These rules cover requests that name a model directly. With model: "inworld/<router-id>", the router replaces the request's models, ignore, sort and provider with the selected variant's model_selection, so set them on the variant instead.

ignore removes providers and models from the candidates the router would otherwise try: the providers of a model named without a prefix, the models fallbacks, and the auto candidates. It never removes a provider/model you name in model. allow_fallbacks: false limits only the providers tried for the primary model. Check metadata.attempts in the response to see every provider and model the router tried.

Find the providers for a model

Browse models and their providers at inworld.ai/models, or call the List models API. Each entry in its models array has a model and a provider. A model offered by several providers has one entry per provider, and those provider values are what you put in provider.order.

bash
# Providers that offer deepseek-v4-flash
curl -s "https://api.inworld.ai/llm/v1alpha/models?model=deepseek-v4-flash" \
  -H "Authorization: Basic $INWORLD_API_KEY" | jq -r '.models[].provider'

# Every provider in the catalog
curl -s https://api.inworld.ai/llm/v1alpha/models \
  -H "Authorization: Basic $INWORLD_API_KEY" | jq -r '[.models[].provider] | unique[]'

Reach out to support@inworld.ai to request a provider or model to be supported.

Provider configuration

FieldTypeDefaultDescription
orderstring[]-Providers to use, in order. Only the listed providers are tried; providers not in the list are excluded.
allow_fallbacksbooleantrueWhether to fall back to the next provider if the first one fails. When false, only one provider is tried for the primary model. Fallback models still apply: those in models, or, when ignore is set without models, the rest of the catalog. See Use only one provider.

How provider selection works

Without provider.order, providers are tried in the order set by sort. Without either setting, they are tried by latency, fastest first.

When provider.order is specified, only the listed providers are tried, in the exact order listed. Providers not in the list are skipped. sort does not apply to the provider order (but still applies to models fallbacks if specified).

The ignore field applies to providers regardless of whether order is specified.

Examples

These examples show router variants. In a request, send model_id as model, and put the same provider object in extra_body.provider, models in extra_body.models, and sort in extra_body.sort as strings (for example, ["price"] instead of [{ "metric": "SORT_METRIC_PRICE" }]).

Default (auto provider selection)
// Automatically selects the lowest-latency provider for deepseek-v4-flash
// Falls back to next-best provider if it fails
{
  "variant_id": "auto-provider",
  "model_id": "deepseek-v4-flash"
}

Execution order

When using provider routing with model fallbacks, the full execution order is:

  1. Try providers for the primary model in provider.order order (if specified) or sorted by sort criteria (default: latency). With allow_fallbacks: false, only the first provider is tried.
  2. If all providers fail and models is specified, fall back to the models list, in the order listed. If sort is set, the fallback models are sorted by it instead. If ignore is set without models, fall back to the rest of the catalog instead, minus what ignore excludes.
  3. If all models fail, return an error.

Use Cases

  • Reliability: Ensure your application continues working even if a specific provider is down
  • Cost Optimization: Route to cheaper providers or fall back to cheaper models
  • Performance: Prefer low-latency providers, or fall back to faster models for time-sensitive requests
  • Provider Control: Lock to specific providers for compliance or consistency