Third-Party STT Models

The Inworld STT API can route transcription to select third-party providers. This is offered for customers consolidating existing third-party STT workloads onto the Inworld platform, with the goal of transitioning to inworld/inworld-stt-1 over time. Third-party models use the same authentication, request format, and endpoints as inworld/inworld-stt-1. Switching models is a modelId change on the STT API, or audio.input.transcription.model in a Realtime session.update.

Before you integrate

Model comparison

Language coverage

Speaker diarization

AssemblyAI and Soniox models support speaker diarization on streaming. Set enableSpeakerDiarization: true together with includeWordTimestamps: true; speaker labels are returned on wordTimestamps[].speaker. These flags are STT API-only — they are not session.update fields.

Numbering varies by provider (AssemblyAI starts at 0, Soniox at 1) — don't hardcode a starting index. Some words may arrive without a label; treat those as unattributed. See the Speaker Diarization guide.

Voice Profile

Voice Profile (age, emotion, pitch, vocal style, accent) is an Inworld platform feature that works with every STT model on the API — including third-party models. Enable it with voiceProfileConfig: { "enableVoiceProfile": true } on the STT API, or providerData.stt.voice_profile: true on Realtime; migrating between models does not affect it.

Provider configuration

Every model accepts the standard transcribeConfig fields documented in the API reference — language hints, sample rate, custom vocabulary, turn-taking thresholds, diarization, and word timestamps all work the same way across models. The sections below cover only what is provider-specific: each provider's optional tuning block and a ready-to-use request example.

AssemblyAI (assemblyaiConfig)

WebSocket streaming only; audio must be LINEAR16.

{
  "transcribeConfig": {
    "modelId": "assemblyai/universal-3-5-pro",
    "audioEncoding": "LINEAR16",
    "sampleRateHertz": 16000,
    "language": "en",
    "assemblyaiConfig": {
      "minEndOfTurnSilenceWhenConfident": 500,
      "maxTurnSilence": 3000,
      "vadThreshold": 0.5
    }
  }
}

Soniox (sonioxConfig)

WebSocket streaming only; audio must be LINEAR16.

{
  "transcribeConfig": {
    "modelId": "soniox/stt-rt-v5",
    "audioEncoding": "LINEAR16",
    "sampleRateHertz": 16000,
    "sonioxConfig": {
      "languageHints": ["en", "es"],
      "languageHintsStrict": true,
      "context": {
        "general": { "domain": "healthcare" },
        "text": "Patient intake call for a cardiology clinic.",
        "terms": ["angioplasty", "pheochromocytoma"]
      }
    }
  }
}

Groq (groqConfig)

Sync (file upload) only — no streaming. Accepts LINEAR16, MP3, OGG_OPUS, FLAC, and AUTO_DETECT, with files up to ~16 MB.

{
  "transcribeConfig": {
    "modelId": "groq/whisper-large-v3",
    "audioEncoding": "AUTO_DETECT",
    "language": "en",
    "groqConfig": {
      "temperature": 0.0
    }
  },
  "audioData": { "content": "<base64-audio>" }
}

Equivalent on inworld-stt-1

When migrating to the Inworld model, these are the replacement configurations:

For example, a third-party streaming config with custom vocabulary, turn tuning, diarization, and word timestamps translates to:

{
  "transcribeConfig": {
    "modelId": "inworld/inworld-stt-1",
    "audioEncoding": "LINEAR16",
    "sampleRateHertz": 16000,
    "language": "en",
    "prompts": ["angioplasty", "pheochromocytoma"],
    "endOfTurnConfidenceThreshold": 0.5,
    "inworldSttV1Config": {
      "minEndOfTurnSilenceWhenConfident": 500,
      "maxTurnSilence": 1500,
      "vadThreshold": 0.5
    },
    "enableSpeakerDiarization": true,
    "includeWordTimestamps": true,
    "voiceProfileConfig": { "enableVoiceProfile": true }
  }
}

For Inworld STT, minimum silence gates the first turn-completion check; maximum silence ends the turn even if confidence is insufficient, once the minimum is reached. Both fields accept 20–5000 ms. Keep the minimum less than or equal to the maximum, and test with your audio: equivalent configuration fields do not guarantee identical turn boundaries across models.

For migration support, contact your account team.

Realtime API

The Realtime API is a streaming surface of the same STT models. Send a session.update to pick the model and map provider knobs onto audio.input.transcription and providerData.stt. Partial updates merge. An unrecognized model rejects the whole update (invalid_request_error / invalid_value on session.audio.input.transcription.model).

Accepted here: assemblyai/universal-3-5-pro, assemblyai/u3-rt-pro, soniox/stt-rt-v4 (bare aliases like u3-rt-pro resolve). Not accepted: Groq (sync only) and soniox/stt-rt-v5.

Default turn detection is semantic_vad / eagerness: "medium", which already fills the four turn knobs. Explicit providerData.stt fields override that mapping. See providerData.stt.

Not exposed: languageHintsStrict, Soniox context.*, speaker diarization, word-timestamp flags.

Example: AssemblyAI

{
  "type": "session.update",
  "session": {
    "audio": {
      "input": {
        "transcription": {
          "model": "assemblyai/universal-3-5-pro",
          "language": "en",
          "prompt": "Patient intake call. Vocabulary: angioplasty, pheochromocytoma."
        }
      }
    },
    "providerData": {
      "stt": {
        "min_end_of_turn_silence": 500,
        "max_turn_silence": 3000,
        "vad_threshold": 0.5,
        "voice_profile": true
      }
    }
  }
}

Example: Soniox

{
  "type": "session.update",
  "session": {
    "audio": {
      "input": {
        "transcription": {
          "model": "soniox/stt-rt-v4",
          "prompt": "Patient intake call. Vocabulary: angioplasty, pheochromocytoma."
        }
      }
    },
    "providerData": {
      "stt": {
        "language_hints": ["en", "es"],
        "end_of_turn_confidence_threshold": 0.7,
        "max_turn_silence": 2000,
        "voice_profile": true
      }
    }
  }
}