Routing

Provider routing

Route requests across providers for the same model

You can specify a model without a provider prefix (e.g., deepseek-v4-flash instead of deepseek/deepseek-v4-flash), and the API will automatically select a provider for you. Optionally, use the model_selection.provider field in your router config to control how providers are selected.

By default, the provider with the lowest latency is selected, and if it fails, the next best provider is tried automatically.

See all models via the List Models endpoint. Current supported providers include OpenAI, Anthropic, Google Vertex AI, Google AI Studio, DeepInfra, DeepSeek, Mistral, Groq, and xAI. Reach out to support@inworld.ai to request a provider or model to be supported.

Provider configuration

Field	Type	Default	Description
`order`	`string[]`	-	Providers to use, in order. Only the listed providers are tried; providers not in the list are excluded.
`allow_fallbacks`	`boolean`	`true`	Whether to fall back to the next provider if the first one fails. When `false`, only one provider is tried for the primary model. Fallback models still apply: those in `models`, or, when `ignore` is set without `models`, the rest of the catalog.

How provider selection works

When no provider.order is specified, the sort criteria determines the order providers are tried. If no sort is specified either, providers are ordered by latency (fastest first).

When provider.order is specified, only the listed providers are tried, in the exact order listed. Providers not in the list are skipped. sort does not apply to the provider order (but still applies to models fallbacks if specified).

The ignore field applies to providers regardless of whether order is specified.

Examples

Default (auto provider selection)

// Automatically selects the lowest-latency provider for deepseek-v4-flash
// Falls back to next-best provider if it fails
{
  "variant_id": "auto-provider",
  "model_id": "deepseek-v4-flash"
}

Explicit provider order

// Try deepseek first, then inworld. No other provider is used.
{
  "variant_id": "deepseek-preferred",
  "model_id": "deepseek-v4-flash",
  "model_selection": {
    "provider": {
      "order": ["deepseek", "inworld"]
    }
  }
}

Sort providers by price

// Pick the cheapest provider for deepseek-v4-flash
// Fall back to next-cheapest if it fails
{
  "variant_id": "cheapest-provider",
  "model_id": "deepseek-v4-flash",
  "model_selection": {
    "provider": {
      "allow_fallbacks": true
    },
    "sort": [{ "metric": "SORT_METRIC_PRICE" }]
  }
}

Provider + model fallbacks

// Try deepseek-v4-flash on deepseek, then on inworld,
// then fall back to gpt-5.4
{
  "variant_id": "with-fallbacks",
  "model_id": "deepseek-v4-flash",
  "model_selection": {
    "provider": {
      "order": ["deepseek", "inworld"],
      "allow_fallbacks": true
    },
    "models": ["openai/gpt-5.4"]
  }
}

No provider fallbacks

// Try the lowest latency provider only - return error if it fails
// (unless fallback models are listed in model_selection.models)
{
  "variant_id": "single-provider",
  "model_id": "deepseek-v4-flash",
  "model_selection": {
    "provider": {
      "allow_fallbacks": false
    }
  }
}

Filter by provider

// Restrict auto-routing to only use models from specific providers.
// A bare provider name in models expands to all of that provider's models.
{
  "variant_id": "openai-only",
  "model_id": "auto",
  "model_selection": {
    "models": ["openai"]
  }
}

Execution order

When using provider routing with model fallbacks, the full execution order is:

Try providers for the primary model in provider.order order (if specified) or sorted by sort criteria (default: latency). With allow_fallbacks: false, only the first provider is tried.
If all providers fail and models is specified, fall back to the models list, sorted by sort criteria.
If all models fail, return an error.

Use Cases

Reliability: Ensure your application continues working even if a specific provider is down
Cost Optimization: Route to cheaper providers or fall back to cheaper models
Performance: Prefer low-latency providers, or fall back to faster models for time-sensitive requests
Provider Control: Lock to specific providers for compliance or consistency