Prompt compression

Prompt compression automatically shortens long messages before they reach the LLM. This reduces the number of input tokens without meaningfully changing the response quality — saving cost on every request.

Compression works best on long system prompts, detailed instructions, and context-heavy messages. Text shorter than 250 tokens is skipped automatically. With variant- or request-level compression, a short message next to a long one can still be compressed.

Compression may also be skipped, without a warning, for very low-cost models, where the savings would be negligible.

Router-level compression

Set compression on a router variant to compress every request routed to that variant. This is the most common setup - configure once, and every request through this variant benefits from compression.

{
  "name": "routers/my-router",
  "display_name": "Compressed prompts",
  "default_route": {
    "route_id": "default",
    "variants": [{
      "weight": 100,
      "variant": {
        "variant_id": "v1",
        "model_id": "openai/gpt-4o",
        "compression": {
          "aggressiveness": 0.7
        },
        "message_templates": [
          {
            "role": "system",
            "content": "You are a helpful assistant with deep expertise in world history, geopolitics, and international relations. Your primary responsibility is to provide detailed, engaging answers..."
          }
        ]
      }
    }]
  }
}

Request-level compression

To compress a single request the same way, pass compression at the top level of the request body (or inside extra_body). It takes the same aggressiveness parameter and overrides any variant-level setting. Like variant-level compression, it applies to all compressible messages and replaces per-message settings.

{
  "model": "openai/gpt-4o",
  "messages": [
    { "role": "system", "content": "You are a support agent for Acme. Follow these policies: ..." },
    { "role": "user", "content": "How do I reset my password?" }
  ],
  "compression": { "aggressiveness": 0.6 }
}

Message-level compression

You can compress specific messages in a chat completion request by adding a compression field to any message. Consecutive messages with the same aggressiveness are batched and compressed together. Non-consecutive compressed messages are processed separately.

curl -X POST https://api.inworld.ai/v1/chat/completions \
  -H "Authorization: Basic $INWORLD_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openai/gpt-4o",
    "messages": [
      {
        "role": "system",
        "content": "You are an expert assistant with extensive knowledge of...",
        "compression": { "aggressiveness": 0.5 }
      },
      {
        "role": "user",
        "content": "What is the capital of France?"
      }
    ]
  }'

Only messages with the compression field are compressed. Other messages are sent as-is. Per-message settings apply only when neither the variant nor the request sets compression.

Response

Non-streaming responses include compression stats in metadata.compression. Streamed responses do not carry compression stats or warnings.

{
  "metadata": {
    "compression": {
      "original_tokens": 190,
      "compressed_tokens": 102,
      "saved_tokens": 88
    }
  }
}

If no messages were compressed, the compression field is omitted from the response.

Warnings

When compression is enabled but skipped or fails for specific messages, a non-streaming response includes warnings in metadata.compression_warnings:

{
  "metadata": {
    "compression_warnings": [
      "Prompt compression skipped for message #0 (system: 'You are a helpful...') — text too short (min 250 tokens)"
    ]
  }
}

Warnings appear when:

• A message has compression enabled but is too short (below 250 tokens)
• The compression service fails for a message (the message is sent uncompressed)