Concurrency limits

Concurrency limits set the guaranteed number of TTS generations your account can run at the same time. They apply to both HTTP and WebSocket requests. Requests above these limits are best-effort based on available capacity.

Default Limits

How Concurrency Is Counted

The way a generation counts toward your limit depends on the protocol:

• HTTP requests: Each in-flight request counts as one concurrent generation. The slot is held for the duration of the request. Exceeding the limit returns a 429 error.
• WebSocket contexts: Each active context counts as one concurrent generation. A context is active while packets are being exchanged, for example send_text from the client or audio chunks from the server. Once a context goes idle, it stops counting toward your limit. See Errors for details on how limit errors are surfaced within a WebSocket connection.

Concurrent Connections

Concurrent connections refer to the number of WebSocket connections your account can maintain simultaneously. This is a separate limit from concurrent generations, set at 10× your concurrent generation limit. Exceeding it returns an HTTP 429 error.

How Many Concurrent Generations Do You Actually Need?

For conversational use cases like voice agents, most applications need far fewer concurrent generations than expected. Because speech is generated much faster than it is spoken, a single generation slot can serve multiple conversations. The diagram below shows three simultaneous conversations and how their generation windows interact. Most generation windows are brief and staggered — only when two overlap does the concurrent generation count increase.

gantt
    title Concurrent Generation Slots Across 3 Simultaneous Conversations
    dateFormat x
    axisFormat %Ss

    section Conversation 1
    User speaking    :a1, 0, 8000
    Generating       :crit, a2, 8000, 10000
    Agent speaking   :a3, 10000, 20000
    User speaking    :a4, 20000, 27000
    Generating       :crit, a5, 27000, 29000
    Agent speaking   :a6, 29000, 38000

    section Conversation 2
    Agent speaking   :b1, 0, 12000
    User speaking    :b2, 12000, 20000
    Generating       :crit, b3, 20000, 22000
    Agent speaking   :b4, 22000, 38000

    section Conversation 3
    User speaking    :c1, 0, 6000
    Generating       :crit, c2, 6000, 9000
    Agent speaking   :c3, 9000, 19000
    User speaking    :c4, 19000, 26000
    Generating       :crit, c5, 26000, 29000
    Agent speaking   :c6, 29000, 38000

    section Concurrent Slots Counted
    1 concurrent generation  :done, d1, 6000, 8000
    2 concurrent generations :crit, d2, 8000, 9000
    1 concurrent generation  :done, d3, 9000, 10000
    1 concurrent generation  :done, d4, 20000, 22000
    1 concurrent generation  :done, d5, 26000, 27000
    2 concurrent generations :crit, d6, 27000, 29000

The red 2 slots counted periods are the moments where two generation windows overlap — only these count as 2 concurrent generations. Outside those windows, a single slot handles all three conversations. We typically find that the number of simultaneous conversations you can handle is at least 4x your concurrent generation limit. Exact numbers will vary depending on your use case, conversation patterns, and response lengths.

Idle WebSocket Contexts

A WebSocket TTS context counts toward your limit only while it is active: while packets are being exchanged, such as send_text from the client or audio chunks from the server. Closing a context releases its slot immediately once all audio chunks have been returned. If a context is left open, it becomes idle shortly after activity stops and no longer counts toward your limit.

Errors

When your account exceeds its concurrent generation limit, the affected request is rejected with an error JSON object containing code: 8:

{
  "error": {
    "code": 8,
    "message": "request failed: rpc error: code = ResourceExhausted desc = maximum allowed number of active WebSocket TTS contexts: 5 is reached",
    "details": []
  }
}

Your WebSocket connection and other active contexts are unaffected.

• If you close and reopen contexts per agent turn, the error occurs on create_context — the context is not created. Any messages sent after this create_context may have been dropped.
• If you keep contexts open while idle, the error can occur on any message that resumes activity (e.g., send_text, flush). Any messages sent since the first send_text after idle may have been dropped.

For retry strategies when hitting rate limits, see Handling rate-limited requests.