Skip to main content

API

Capabilities configuration

const capabilities = new Capabilities({ emotions: true });
NameDescriptionDefault value
audioIf false, then the client will not receive spoken audio from the characters (text only mode).true
emotionsYou will receive character emotions.false
interruptionsAllow interruptions to conversations with the character.false
phonemesInclude phoneme information in Audio Events.false
silenceAllow for pauses between character replies.false
narratedActionsAllow to receive information about character narrated actionsfalse

Connection configuration

const connection = {
disconnectTimeout: 10 * 1000, // time in milliseconds
autoReconnect: false, // true by default
}
NameDescriptionDefault value
disconnectTimeoutClose connection due to inactivity60000
autoReconnectConnection will be opened automatically on send if it is closed. Otherwise, an existing open connection will be used. Our server closes the connection automatically after 1 minute of inactivity.true

SessionToken

const token = client.generateSessionToken();

// Get a token.
token.token

// Get type (Bearer).
token.type

// Get expiration time.
// Token should be regenerated after expiration time.
token.expirationTime

// Get session id.
token.sessionId

Token lifetime duration is 1 hour by default, but note that there is also a session lifetime on the Server side, which is 30 minutes (if you stop talking/chatting).

Sessions are thus expired when there is at least 30 minutes of inactivity. In this case you will have to recreate the connection to start a new session.

However if the token is expired but the session is still alive, Node.JS SDK will reload the token automatically. See Connection configuration above to change default values.

InworldClient with API key

If you prefer to use automatic session management set client key/secret.

const client = new InworldClient();

// ApiKey is required.
client.setApiKey({ key, secret });

// User is not required.
client.setUser(user);

// Configuration is not required.
client.setConfiguration({
capabilities,
connection: {
disconnectTimeout: 10 * 1000, // time in milliseconds
autoReconnect: false,
}
});

// Scene is not required for token generation.
// But if you would like to speak with scene characters you need to provide scene full name.
// It should be like workspaces/{WORKSPACE_NAME}/characters/{CHARACTER_NAME}.
// Or like workspaces/{WORKSPACE_NAME}/scenes/{SCENE_NAME}.
client.setScene(scene);

// Event handlers
client.setOnDisconnect(fn);
client.setOnError(fn);
client.setOnWarning(fn); // If no function is specified, console.warn will be invoked.
client.setOnMessage(fn);

// Generate SessionAccessToken.
client.generateSessionToken();

// Finish connection configuration.
// Return instance of EventService.
// Сonnection is not open. It's just ready to open on message send.
const connection = client.build();

Unitary Session

Starting from version 1.12.0, our server-side supports "Unitary Session," enabling us to perform several operations without the need to restart the session.

Reload or change scene

You can dynamically alter scene names during runtime without the need to recreate connections. Furthermore, you have the capability to reload the current scene, either to remove manually added characters or to initiate the session anew from the beginning.

  // Reload current scene.
await connection.reloadScene();

// Scene name should be like workspaces/{WORKSPACE_NAME}/characters/{CHARACTER_NAME}.
// Or like workspaces/{WORKSPACE_NAME}/scenes/{SCENE_NAME}.
await connection.changeScene(sceneName);

Add Characters

Once you initiate the connection and designate the scene name, only characters associated with that scene are accessible for conversation. However, you have the flexibility to introduce additional character(s) into the conversation without having to include them in the scene via the Studio UI interface.

  const scene = 'workspaces/test-workspace/scenes/test-scene';
const names = [
'workspaces/test-workspace/characters/character-1',
'workspaces/test-workspace/characters/character-2',
'workspaces/test-workspace/characters/character-3',
]

const client = new InworldClient();

client.setScene(scene);

const connection = client.build();

...

await connection.addCharacters(names);

Session management

Session is expired after 30 minutes of inactivity, resulting in the loss of previous conversation history. Nonetheless, there's a solution to preserve this history.

Previos dialog

You can store all text messages in a location of your preference, allowing you to transfer saved messages to a new session.

For instance, consider the following conversation:

Player: 'Hi'
Character: 'Hi, Player.'
Character: 'How can I assist you today?'

You can save it in an array like this:

const previousDialog = [
{
talker: DialogParticipant.PLAYER,
phrase: 'Hi',
},
{
talker: DialogParticipant.CHARACTER,
phrase: 'Hi, Player. How can I assist you today?',
},
];

To maintain this conversation after the session has expired, you should establish a new connection. Simply use the setSessionContinuation method and provide the previousDialog to new connection as shown below:

import { InworldClient, DialogParticipant } from '@inworld/nodejs-sdk';

const client = new InworldClient();

client.setSessionContinuation({ previousDialog });

Previous state

You can retrieve the conversation state from the Inworld AI server using the following method:

import { InworldClient } from '@inworld/nodejs-sdk';

let previousState: string;
const client = new InworldClient();

client.setOnDisconnect(async () => {
previousState = (await connection.getSessionState()).state;
});

If you'd like to carry on the conversation with the previous state after the session has ended, you'll need to establish a new connection. Simply use the setSessionContinuation method for new connection and propagate the previousState as demonstrated below:

import { InworldClient } from '@inworld/nodejs-sdk';

const client = new InworldClient();

client.setSessionContinuation({ previousState });

InworldConnectionService

const connection = client.build();
// Open connection manually. Available only if configuration.connection.autoReconnect = false.
// Otherwise connection will be managed automatically by SDK.
connection.open();
// You can check if the connection is open or not in case of configuration.connection.autoReconnect = false.
connection.isActive();
// Send text message.
connection.sendText(message: string);
// Send trigger. Pass trigger name and parameters as arguments.
interface TriggerParameter {
name: string;
value: string;
}
connection.sendTrigger(name: string, parameters?: TriggerParameter[]);
// Send narrated action.
connection.sendNarratedAction(text: string);
// Send audio start event before call sendAudio.
connection.sendAudioSessionStart();
// Send audio end event after all audio chunks you would like to send.
connection.sendAudioSessionEnd();
// Send audio. Pass string chunk as argument.
connection.sendAudio(chunk);
// Send a cancel response.
// InteractionId or utteranceId can be omitted.
// When interactionId is empty, everything in the session will be removed.
// When only the utteranceId is provided, nothing happens.
// When only the interactionId is provided, everything until this interaction will be removed.
// When both the interactionId and utteranceId are provided, everything until this interaction will be removed, and utterances in this interaction will also be removed.
connection.sendCancelResponse({
interactionId?: string,
utteranceId?: string[],
});
// Close connection.
connection.close();
// Get a character list.
connection.getCharacters();
// Get the current character.
connection.getCurrentCharacter();
// Change character in the scene.
connection.setCurrentCharacter(character);

User

const user = {
id: 'user-id',
fullName: 'FirstName LastName',
profile: {
fields: [{ id: 'field_1', value: 'value_1' }]
},
};

// Globally unique string, id of the end user of the system.
// UUID will be used by default.
user.id
// User name.
user.fullName
// List of user profile fields.
user.profile.fields

Character

const character = connection.getCurrentCharacter();

// Character id.
character.id

// Character resource name.
character.resourceName

// Character display name.
character.displayName

// Character assets.
character.assets.avatarImg
character.assets.avatarImgOriginal
character.assets.rpmModelUri
character.assets.rpmImageUriPortrait
character.assets.rpmImageUriPosture

InworldPacket

client.setOnMessage((packet: InworldPacket) => {...});

// It's a text event.
packet.isText();

// It's an audio event.
packet.isAudio();

// It's a trigger event.
packet.isTrigger();

// It's an emotion event.
packet.isEmotion();

// It's a control event.
packet.isControl();

// It's a silence event.
packet.isSilence();

// It's a cancel response event.
packet.isCancelResponse();

// It's a special control event. It means that interaction ends.
packet.isInteractionEnd();

client.setOnMessage defines an event listener for incoming messages(packets) with different types of events using the provided methods.

  1. client.setOnMessage((packet: InworldPacket) => {...}); - This line sets up an event listener for the client object. The event listener listens for incoming messages (packets) of type InworldPacket and processes them using the provided callback functions.

Within the callback function, the packet object has several methods to check for specific types of events in the received packet.

Common Event Data

// ISO string.
packet.date


// A token that uniquely identifies the packet.
packet.packetId.packetId
// A token that uniquely identifies and groups utterances in the replies.
// Different packets may belong to the same utterance. E.g. Audio Event and Text Event of the same spoken utterance.
packet.packetId.utteranceId
// A token that uniquely identifies interaction between actors.
packet.packetId.interactionId

// Determines who sends the packet: player or character.
packet.routing.source.name
packet.routing.source.isPlayer
packet.routing.source.isCharacter

// Determines who receives the packet: player or character.
packet.routing.target.name
packet.routing.target.isPlayer
packet.routing.target.isCharacter

Text Event

// Get message of text event.
packet.text.text
// If this is the final version of the text or not.
// For instance speech recognition may take some time to finalize the text.
packet.text.final

Audio Event

// Get chunk of audio event.
packet.audio.chunk

// Get list if phonemes.
packet.audio.additionalPhonemeInfo = [];

// Get phoneme data.
// Synthesized phoneme.
phonemeInfo.phoneme
// Offset from the beginning of audio segment (in seconds).
phonemeInfo.startOffsetS

Emotion Event

// Get behavior affected by emotions.
packet.emotion.behavior.code.NEUTRAL
packet.emotion.behavior.code.DISGUST
packet.emotion.behavior.code.CONTEMPT
packet.emotion.behavior.code.BELLIGERENCE
packet.emotion.behavior.code.DOMINEERING
packet.emotion.behavior.code.CRITICISM
packet.emotion.behavior.code.ANGER
packet.emotion.behavior.code.TENSION
packet.emotion.behavior.code.TENSE_HUMOR
packet.emotion.behavior.code.DEFENSIVENESS
packet.emotion.behavior.code.WHINING
packet.emotion.behavior.code.SADNESS
packet.emotion.behavior.code.STONEWALLING
packet.emotion.behavior.code.INTEREST
packet.emotion.behavior.code.VALIDATION
packet.emotion.behavior.code.AFFECTION
packet.emotion.behavior.code.HUMOR
packet.emotion.behavior.code.SURPRISE
packet.emotion.behavior.code.JOY

// Get strength of the emotion.
packet.emotion.strength.code.UNSPECIFIED
packet.emotion.strength.code.WEAK
packet.emotion.strength.code.STRONG
packet.emotion.strength.code.NORMAL

Trigger Event

// Trigger name.
packet.trigger.name
// Parameters that come with given event.
packet.parameters

// Parameter name.
parameter.name
// Parameter value.
parameter.value

Control Event

packet.control.type.INTERACTION_END
packet.control.type.WARNING

Silence Event

// Silence duration in milliseconds.
packet.silence.durationMs

Cancel Response Event

// Interaction id for response cancellation.
packet.cancelResponses.interaction_id
// Utterance ids to cancel for the given interaction id.
packet.cancelResponses.utterance_id

Interaction End

Interaction End is a type of Control Event to determine when the Character has ended responding to a sent message. It is considered ended when the chat has not started ( there is no actor action yet ) or the last received message is INTERACTION_END.

onMessage: (inworldPacket: InworldPacket) => {
if (inworldPacket.isInteractionEnd()) {
// Handle end of response
}
}

Determining which interaction has ended for multiple sent messages can be done by comparing the interactionId, returned when a Player sends a message, to the interactionId received from a responding message event.

const sendPacketA = await connection.sendText('Hi');
const sendPacketB = await connection.sendText('How are you?');
onMessage: (inworldPacket: InworldPacket) => {
if (inworldPacket.isInteractionEnd()) {
if (sendPacketA.packetId.interactionId == inworldPacket.packetId.interactionId) {
// Handle end of response for sent messsage 'Hi'
}
if (sendPacketB.packetId.interactionId == inworldPacket.packetId.interactionId) {
// Handle end of response for sent messsage 'How are you?'
}
}
}

Feedback API

To share your feedback with us, you can utilize the following API calls:

import {
DislikeType,
} from '@inworld/nodejs-sdk';


// To send like.
const like = connection.feedback.like({
interactionId, // Required.
correlationId, // Optional. Used for response regeneration.
});

// To send dislike.
const dislike = connection.feedback.dislike({
comment: 'I do not like this response', // Optional.
interactionId, // Required.
correlationId, // Optional. Used for response regeneration.
types: [DislikeType.IRRELEVANT], // Optional.
});

// To undo a like or dislike made by mistake.
connection.undo(like.name);
connection.undo(dislike.name);

List of Dislike Types

DislikeType.IRRELEVANT
DislikeType.UNSAFE
DislikeType.UNTRUE
DislikeType.INCORRECT_USE_KNOWLEDGE
DislikeType.UNEXPECTED_ACTION
DislikeType.UNEXPECTED_GOAL_BEHAVIOR
DislikeType.REPETITION
DislikeType.UNSPECIFIED

Error handling

import {
status,
} from '@inworld/nodejs-sdk';

handleError(err: ServiceError) {
switch (err.code) {
// Cancelled by server due timeout inactivity.
case status.ABORTED:
// Cancelled by client.
case status.CANCELLED:
break;
default:
console.error(err);
break;
}
};

You can read more about possible GRPC status codes here.

Environment

Use these environment variables for local debugging of the package.

NameDescriptionDefault value
NODE_SDK_INWORLD_ENGINE_HOSTAPI endpointapi-engine.inworld.ai:443
NODE_SDK_INWORLD_ENGINE_SSLDefine if Engine connection uses TLStrue
NODE_SDK_INWORLD_LOGGER_LEVELPossible values: debug, warn, error
NODE_SDK_INWORLD_LOGGER_FILEFile path for logs