Skip to main content

Audio Input Modes

Since the release of the Inworld Unity SDK 3.1, we have integrated Acoustic Echo Cancellation (AEC) functionality and offer various audio input modes.

Users can select their preferred mode within the InworldController's AudioCapture or InworldAECAudioCapture scripts.

AudioInputMethod

AEC​

AEC mode is our current default audio input method, designed to diminish the echo of the character's voice being fed back into the microphone. We utilize a DLL written in Native C++ within the NDK for echo cancellation.

Notes:​

  1. AEC is currently supported only on Windows and Mac, and it is available exclusively in the full SDK, not within the Core package of inworld-unity. You can find the corresponding DLL under Assets/Inworld/Inworld.NDK/Plugins/.

AECDll

  1. We will not be adding AEC support for Android, iOS, or other XR devices, as these products generally offer hardware-level AEC support.

  2. If your SDK or the corresponding platform does not support AEC, we will automatically fallback to the Turn-based mode.

protected override void Init()
{
    if (IsAvailable)
    {
        AudioConfiguration audioSetting = AudioSettings.GetConfiguration();
        audioSetting.speakerMode = AudioSpeakerMode.Mono;
        audioSetting.sampleRate = k_SampleRate; // 16000
        AudioSettings.Reset(audioSetting);
        m_AECHandle = AECInterop.WebRtcAec3_Create(k_SampleRate);
    }
    else
        m_SamplingMode = MicSampleMode.TURN_BASED;
    m_LastSampleMode = m_SamplingMode;
    base.Init();
}

Dumping Audio​

We acknowledge that the quality of the AEC plugin is not yet perfect, and we are committed to improving it. Starting with version 3.3, we have introduced an audio dumping feature that allows you to generate wave files containing both raw and processed audio data for testing purposes.

To use this feature, you must assign a hotkey for dumping audio.

DumpAudio

When the game is running, HOLD the assigned hotkey to start recording the audio.

To stop recording and generate the wave files, RELEASE the hotkey.

Those files are parallel to your project. You can then listen to these files to compare the differences between the raw and processed audio.

DumpFile

Turn based​

This mode is the default fallback when AEC is not available. In this mode, players cannot speak while any character is speaking.

Players can fast-forward through a character's speech, which, by default in the SDK, is achieved by holding down the Ctrl key. For details, please refer to the Text Ctrl documentation.

No Mic​

This mode is automatically selected when either the AEC DLL or the microphone malfunctions, rendering the microphone unavailable.

No Filter​

Selecting this mode disables AEC, and players can also interrupt a character's speech.

Push to talk​

In Push to Talk mode, players must press and hold a specific key to speak. By default in the SDK, entering the global chat panel by pressing the ~ key automatically switches the system to Push to Talk mode. Users can then press and hold the Record button to record audio, releasing the key to send the recording.

PTT