Speaker Diarization

Speaker diarization identifies who said what in multi-speaker audio — calls, meetings, interviews — by labeling each transcribed word with a speaker identifier.

Enabling diarization

Diarization is available on the WebSocket streaming endpoint. Set both fields in transcribeConfig:

{
  "transcribeConfig": {
    "modelId": "assemblyai/u3-rt-pro",
    "audioEncoding": "LINEAR16",
    "sampleRateHertz": 16000,
    "enableSpeakerDiarization": true,
    "includeWordTimestamps": true
  }
}

Set includeWordTimestamps together with enableSpeakerDiarization — speaker labels are attached to word timestamps, not to the turn-level transcript.

Reading speaker labels

Each word in wordTimestamps carries a speaker identifier:

{
  "result": {
    "transcription": {
      "transcript": "Hi, I'm calling about my account balance.",
      "isFinal": true,
      "wordTimestamps": [
        { "word": "Hi,", "startTimeMs": 97, "endTimeMs": 194, "confidence": 0.99, "speaker": 0 },
        { "word": "I'm", "startTimeMs": 698, "endTimeMs": 779, "confidence": 0.99, "speaker": 0 }
      ]
    }
  }
}

Speaker identifiers are integers assigned within a stream in order of first appearance. They mark “same speaker in this session” — they do not identify a person across sessions.

Numbering varies by provider: AssemblyAI starts at 0, Soniox at 1.

To build per-speaker segments, group consecutive words that share a speaker value. Words may occasionally arrive without a label; treat those as unattributed rather than assuming a speaker.

Model support

Model	Speaker diarization
`assemblyai/u3-rt-pro`
`assemblyai/universal-streaming-english`
`assemblyai/universal-streaming-multilingual`
`soniox/stt-rt-v4`
`soniox/stt-rt-v5`
`deepgram/flux-general-en`
`deepgram/flux-general-multi`
`inworld/inworld-stt-1`	Coming soon
`assemblyai/whisper-rt`
`groq/whisper-large-v3`

Enabling diarization

Reading speaker labels

Model support

Next steps

WebSocket API Reference

Turn Detection

​Enabling diarization

​Reading speaker labels

​Model support

​Next steps

WebSocket API Reference

Turn Detection

Enabling diarization

Reading speaker labels

Model support

Next steps