Multi-Agent

Inworld’s technology provides support for multiple AI characters talking to each other, referred to as Multi-Agent conversations. This guide covers what Multi-Agent is, how it works, and how to use it to add NPC conversations to your game.

Unity Multi-Agent Demo

For an example of how Multi-Agent system in action in unity, see: Unity Multiple Characters Demo

How Multi-Agent Conversations Work

With Inworld’s Multi-Agent feature, it is possible to have from 2 to 5 AI-powered characters talking either (1) amongst themselves, or (2) with the player in a group conversation.

This feature can be tested within Inworld Studio, and in Inworld’s integrations for Unity, Unreal Engine, and WebSDK.

Conversations

A Conversation is comprised of a list of between 2 to 6 Participants.

Participants are actors classified as either an Agent or a Player.

• An Agent is an AI-controlled character.
• A Player is a human-controlled character.

A conversation can a maximum of 5 Agent participants. Agents added to a conversation must be present in the current Scene.

In the event of a Scene Load, agents not present in the loaded scene are removed from all active conversations.

Conversation Initiation

Clients initiate a conversation by sending a CONVERSATION_UPDATE control event packet with the list of actors. Clients then receive a CONVERSATION_EVENT control event packet every time a conversation is modified.

CONVERSATION_EVENT packets have 3 types of events:

• STARTED: when a conversation is first created.
• UPDATED: when a conversation’s settings (e.g. list of participants) is modified.
• EVICTED: when a conversation is evicted, due to exceeding the max number of allowed conversations.

Conversations Among Agents

Conversations can be comprised solely of Agent participants.

When a conversation is comprised exclusively of Agent participants, dialog between the participants is generated by sending the custom event inworld.conversation.next_turn.

Agents and Players

To cause Agent participants to be aware of the player in the conversation, a Player actor can be included in the conversation participant list.

If the client sends text or audio packets from the Player to the conversation, the Player is automatically added to the conversation’s list of participants.

Conversation ID

Conversations each have an associated Conversation ID.

Including a Conversation ID in the Packet ID indicates that the client wishes to send the packet to the overall conversation, as opposed to a specific Agent.

If the Conversation ID is included, clients must not set a target agent in the packet routing.

Agent Conversational Memory

Agents participating in a conversation retain context on all recent dialog that has been made within that conversation.

Agents are not made aware of any new dialog generated within a conversation after they have left that conversation, but retain context on any dialog that occurred while they were in that conversation.

These Agents will also be aware of any additional dialog generated after they’ve joined a new conversation.

⚠️ There can be a maximum of 20 Conversations active at the same time within a session. If this limit is exceeded, older Conversations are evicted in favor of newer ones. ⚠️

Setting Up Multi-Agent

This tutorial guides you through setting up a basic multi-agent scene in Unity or Unreal Engine.

For additional information on how to get started in Unity/Unreal Engine, see the integration specific documentation:

• Inworld Unreal Engine SDK: Unreal Engine
• Inworld Unity SDK: Unity

Multi-Agent support is also available in Inworld’s Web and Node.js integrations.

Prerequisite

Before beginning this tutorial, ensure you have set up a Workspace containing a Scene with Multiple Characters within Inworld Studio.

Note: In Unreal Engine, it is also possible to add characters to the scene at runtime utilizing the Load Characters function.

Unreal Engine

To set up Multi-Agent for Unreal Engine, complete the steps below.

1. Enable Multi-Agent

Ensure the Multi-Agent capability is enabled on the Inworld Session component within your level.

2. Enable Multiple Targets

If using the Inworld Player Targeting Inworld Player Targeting Component, enable Multiple Targets.

This ensures all valid targets within the interaction distance are added to the conversation.

Alternate Enable

Alternatively, you can utilize the Add Target Character and Remove Target Character functions of the Inworld Player component to manage the characters in the conversation.

Adding multiple target characters results in a multi-agent conversation.

3. Final Steps

Proceed to the Final Steps phase below to complete the setup.

Unity

To set up Multi-Agent for Unity, complete the steps below.

1. Enable Multi-Agent

Ensure the Multi-Agent capability is enabled within the InworldAI asset (this should be enabled by default).

2. Enable Group Chat

Enable Group Chat on the Inworld Client.

3. Set Selecting Method

Then, set the Selecting Method on Character Handler 3D to Auto Chat.

4. Final Steps

Proceed to the Final Steps phase below to complete the setup.

Final Steps

To finish the setup process on both Unreal Engine and Unity, add all your characters to the scene/level.

Now, when you play the scene, you can talk to multiple characters at the same time.

By default in Unreal Engine, if you are utilizing the Inworld Player Targeting Component, a character must be within the Interaction Distance to be in the conversation.

Controlling a Multi-Character Conversation

When the Player speaks to a group of characters in a conversation, Inworld selects a character in the group to respond. This works identically to conversing with a single character.

If you would like the Agents within a conversation to converse amongst themselves without being prompted by the Player, there is a special trigger that can be sent from the client to initiate a response within the group:

inworld.conversation.next_turn

Unity Trigger

Within Unity, this can be sent using the following line:

InworldController.Instance.SendTrigger(InworldMessenger.NextTurn);

Unreal Engine Trigger

From Unreal Engine you can utilize the Continue Conversation function of the Inworld Player component:

Character Selection and Response

When the above trigger is received, Inworld then selects the most suitable character from within the conversation group to respond and generates a new response for that character.

Conversation Handling

Depending if you are developing in Unreal Engine or Unity, the way conversation are handled is done differently, as outlined below.

Unreal Engine

Within Unreal Engine, a single conversation is automatically managed for you by the SDK.

When a character is added as a target or the first time either (1) automatically through the Inworld Player Targeting Component, or (2) manually using the Add Target Character function of the Inworld Player component, a new conversation is generated.

Additional added characters will join this conversation. If at any time the player’s target characters are all removed (e.g. by utilizing the Remove Target Character function), a new conversation is created for the next target.

In Unreal Engine if you wish to have a conversation consisting of only Inworld characters (to prevent player intervention), you can achieve this by un-checking the Conversation Participant Flag of the Inworld Player Component.

Unity

Within Unity, a single conversation is automatically managed for you by the SDK.

When the game starts, a new conversation is generated. By default, all characters in the scene automatically join this conversation.

This conversation can then be managed by the Character Handler, which registers/un-registers characters as needed, which adds/removes them from the conversation.

If you wish to create a new conversation, you can call CharacterHandler.StartNewConversation:

public virtual void StartNewConversation(string conversationID = null) 
    => m_ConversationID = string.IsNullOrEmpty(conversationID) ? Guid.NewGuid().ToString() : conversationID;

Note: Changing the Conversation ID m_ConversationID of Character Handler causes a new conversation to be created.

Limitations of Multi-Agent System

The Multi-Agent conversation system currently has the following limitations:

Cannot Manually Select Characters

There is currently no way to manually select which character in the conversation will speak next. This is controlled by Inworld.