Echo cancellation

Use acoustic echo cancellation (AEC) when your application plays agent audio through speakers while the microphone is open. Speaker audio can enter the microphone, appear in the user's transcript, and trigger an interruption while the agent is still speaking.

Configure AEC in your client's audio pipeline. An echo canceller uses a reference signal, a copy of the audio being played, to identify that audio in the microphone input. Requesting AEC is the first step; playback routing and the browser's implementation also affect the result.

This guide covers browser clients, including mobile browsers, using WebRTC or WebSocket. Native mobile applications need their platform's audio capture and playback configuration.

It is highly recommended to implement AEC in your client. Realtime API does not provide server-side echo cancellation since we cannot track client playback.

Enable echo cancellation on the microphone

Request echo cancellation when you acquire the microphone:

const stream = await navigator.mediaDevices.getUserMedia({
  audio: {
    channelCount: 1,
    echoCancellation: true,
    noiseSuppression: true
  }
});

Send audio from this stream through your existing capture path: add its track to your WebRTC connection, or pass it to your WebSocket client's PCM encoder. Keep the sample rate and encoding required by your session.

noiseSuppression requests background noise reduction. It serves a different purpose from AEC. The browser decides which playback sources to cancel when echoCancellation is true; see the media capture specification. A track reporting echoCancellation: true in getSettings() does not measure how much echo remains.

Check the playback path

Keep one playback path for each agent audio stream. Test the capture and playback paths together on the browsers and devices your application supports.

WebRTC sessions

Play the remote audio track through an <audio> element, as shown in the WebRTC quickstart. This keeps playback on the browser's WebRTC media path:

flowchart LR
    A["Remote WebRTC audio track"] --> B["audio element"] --> C["Speaker"]

If echo persists, check whether your application also plays the agent audio through a separate Web Audio graph. Test with only the remote track playing through the media element.

WebSocket sessions

WebSocket clients decode and schedule received audio themselves. If direct playback through AudioContext.destination produces echo, try routing the final playback node through a MediaStreamAudioDestinationNode into an <audio> element. Inworld's demo client uses this route on mobile browsers.

flowchart LR
    A["Decoded PCM playback graph"] --> B["MediaStreamAudioDestinationNode"] --> C["audio element"] --> D["Speaker"]

Use the existing playback AudioContext, named ctx below. Create one destination and one playback element per session before requesting the first reply. Keep them in your session state so the code that schedules audio chunks can use them. Start or resume playback from your application's start button:

const playback = ctx.createMediaStreamDestination();
const speaker = document.createElement('audio');
speaker.autoplay = true;
speaker.srcObject = playback.stream;
document.body.appendChild(speaker);

await ctx.resume();
await speaker.play();

Connect the final node of your playback graph to playback. For example, replace src.connect(ctx.destination) with src.connect(playback) in the WebSocket quickstart. If you use a gain or analyser node after src, change that node's output connection instead. Keep your existing decoding, buffering, and scheduling.

Remove the previous connection to ctx.destination so audio plays once. If play() rejects, show a control that retries playback from a user action. On session teardown, stop queued audio and microphone tracks, disconnect the graph, clear speaker.srcObject, remove the element, and close any audio contexts owned by that session.

This routing is an approach to test when troubleshooting echo. It does not guarantee cancellation on every browser, operating system, or output device.

Consider a local WebRTC loopback

If echo persists with WebSocket playback, a local WebRTC loopback is another approach to test. Two RTCPeerConnection objects exchange audio within the page so playback arrives as a remote WebRTC track, which the browser can use as an echo reference. The connection to the Realtime API remains WebSocket.

Wire the streams as follows:

Encode the microphone stream received by peer A. Keep the raw microphone stream as the input to peer B.

Before switching to loopback, wait for audio tracks on both receiving peers. Use a setup timeout and restore the direct capture and media-element playback paths if setup fails. Close both peers on failure or session teardown. The browser's RTCPeerConnection reference covers track delivery and connection setup.

Enable this path only on browser and device combinations you have tested. Inworld's demo uses direct media-element playback on iOS and skips loopback there. Use that as a starting point for your own device tests.

Verify echo reduction and interruptions

Test with microphone capture enabled throughout agent playback:

1. Let a reply play while you stay silent. Check for echoed words in input transcripts and unexpected input_audio_buffer.speech_started events.
2. Speak while the reply is playing. Check that your speech is captured and that interruption handling still stops the reply as intended.
3. Repeat at normal and high speaker volumes, then with headphones or Bluetooth. Repeat after changing the audio route or restarting a session.
4. Record the browser, operating system, output device, and active playback path alongside failures. Compare direct playback and loopback where available.

Muting the microphone during playback can help identify speaker leakage, but it also prevents spoken interruptions. Increasing turn detection thresholds or disabling interrupt_response changes interruption behavior; neither establishes that echo has been removed from the microphone input.