API

Capabilities configuration

const capabilities = new Capabilities({ emotions: true });

Name	Description	Default value
audio	If false, then the client will not receive spoken audio from the characters (text only mode).	true
emotions	You will receive character emotions.	false
interruptions	Allow interruptions to conversations with the character.	false
phonemes	Include phoneme information in Audio Events.	false
silence	Allow for pauses between character replies.	false

Connection configuration

const connection = {
  disconnectTimeout: 10 * 1000, // time in milliseconds
  autoReconnect: false, // true by default
}

Name	Description	Default value
disconnectTimeout	Close connection due to inactivity	60000
autoReconnect	Connection will be opened automatically on send if it is closed. Otherwise, an existing open connection will be used. Our server closes the connection automatically after 1 minute of inactivity.	true

SessionToken

const token = client.generateSessionToken();

// Get a token.
token.token

// Get type (Bearer).
token.type

// Get expiration time.
// Token should be regenerated after expiration time.
token.expirationTime

// Get session id.
token.sessionId

InworldClient with API key

If you prefer to use automatic session management set client key/secret.

const client = new InworldClient();

// ApiKey is required.
client.setApiKey({ key, secret });

// User is not required.
client.setUser(user);

// Configuration is not required.
client.setConfiguration({
  capabilities,
  connection: {
    disconnectTimeout: 10 * 1000, // time in milliseconds
    autoReconnect: false,
  }
});

// Scene is not required for token generation.
// But if you would like to speak with scene characters you need to provide scene full name.
// It should be like workspaces/{WORKSPACE_NAME}/characters/{CHARACTER_NAME}.
// Or like workspaces/{WORKSPACE_NAME}/scenes/{SCENE_NAME}.
client.setScene(scene);

// Event handlers
client.setOnDisconnect(fn);
client.setOnError(fn);
client.setOnMessage(fn);

// Generate SessionAccessToken.
client.generateSessionToken();

// Finish connection configuration.
// Return instance of EventService.
// Сonnection is not open. It's just ready to open on message send.
const connection = client.build();

Manage session manually

If you prefer to manage a session manually you should provide a onSessionGet and onSessionSet functions. This will allow you to store session externally (for example, in redis) and reuse a session on application restart. See the Discord bot example for more details.

import { Session } from '@inworld/nodejs-sdk';

const key = 'character_key';

async function onSessionGet() {
  const json = await redisClient.get(key);

  return Session.deserialize(json);
}


function onSessionSet(key: string, entity: Session) {
  redisClient.set(key, Session.serialize(entity));
}

const client = new InworldClient();

// User is not required.
client.setUser(user);

// Configuration is not required.
client.setConfiguration({
  capabilities,
  connection: {
    disconnectTimeout: 10 * 1000, // time in milliseconds
    autoReconnect: false,
  }
});

// It should be like workspaces/{WORKSPACE_NAME}/characters/{CHARACTER_NAME}.
// Or like workspaces/{WORKSPACE_NAME}/scenes/{SCENE_NAME}.
client.setScene(scene);

// Event handlers
client.setOnDisconnect(fn);
client.setOnError(fn);
client.setOnMessage(fn);

// Allow session data to be kept in some storage, such as Redis, and reused.
// This reduces the number of API calls if the session has not expired.
client.setOnSession({
  get: onSessionGet,
  set: onSessionSet,
});

// Finish connection configuration.
// Return instance of EventService.
// Сonnection is not open. It's just ready to open on message send.
const connection = client.build();

InworldConnectionService

const connection = client.build();
// Open connection manually. Available only if configuration.connection.autoReconnect = false.
// Otherwise connection will be managed automatically by SDK.
connection.open();
// You can check if the connection is open or not in case of configuration.connection.autoReconnect = false.
connection.isActive();
// Send text message.
connection.sendText(message: string);
// Send trigger. Pass trigger name and parameters as arguments.
interface TriggerParameter {
  name: string;
  value: string;
}
connection.sendTrigger(name: string, parameters?: TriggerParameter[]);
// Send audio start event before call sendAudio.
connection.sendAudioSessionStart();
// Send audio end event after all audio chunks you would like to send.
connection.sendAudioSessionEnd();
// Send audio. Pass string chunk as argument.
connection.sendAudio(chunk);
// Send a cancel response.
// InteractionId or utteranceId can be omitted.
// When interactionId is empty, everything in the session will be removed.
// When only the utteranceId is provided, nothing happens.
// When only the interactionId is provided, everything until this interaction will be removed.
// When both the interactionId and utteranceId are provided, everything until this interaction will be removed, and utterances in this interaction will also be removed.
connection.sendCancelResponse({
  interactionId?: string,
  utteranceId?: string[],
});
// Close connection.
connection.close();
// Get a character list.
connection.getCharacters();
// Get the current character.
connection.getCurrentCharacter();
// Change character in the scene.
connection.setCurrentCharacter(character);

Character

const character = connection.getCurrentCharacter();

// Character id.
character.id

// Character resource name.
character.resourceName

// Character display name.
character.displayName

// Character assets.
character.assets.avatarImg
character.assets.avatarImgOriginal
character.assets.rpmModelUri
character.assets.rpmImageUriPortrait
character.assets.rpmImageUriPosture

InworldPacket

client.setOnMessage((packet: InworldPacket) => {...});

// It's a text event.
packet.isText();

// It's an audio event.
packet.isAudio();

// It's a trigger event.
packet.isTrigger();

// It's an emotion event.
packet.isEmotion();

// It's a control event.
packet.isControl();

// It's a silence event.
packet.isSilence();

// It's a cancel response event.
packet.isCancelResponse();

// It's a special control event. It means that interaction ends.
packet.isInteractionEnd();

Common Event Data

// ISO string.
packet.date


// A token that uniquely identifies the packet.
packet.packetId.packetId
// A token that uniquely identifies and groups utterances in the replies.
// Different packets may belong to the same utterance. E.g. Audio Event and Text Event of the same spoken utterance.
packet.packetId.utteranceId
// A token that uniquely identifies interaction between actors.
packet.packetId.interactionId

// Determines who sends the packet: player or character.
packet.routing.source.name
packet.routing.source.isPlayer
packet.routing.source.isCharacter

// Determines who receives the packet: player or character.
packet.routing.target.name
packet.routing.target.isPlayer
packet.routing.target.isCharacter

Text Event

// Get message of text event.
packet.text.text
// If this is the final version of the text or not.
// For instance speech recognition may take some time to finalize the text.
packet.text.final

Audio Event

// Get chunk of audio event.
packet.audio.chunk

// Get list if phonemes.
packet.audio.additionalPhonemeInfo = [];

// Get phoneme data.
// Synthesized phoneme.
phonemeInfo.phoneme
// Offset from the beginning of audio segment (in seconds).
phonemeInfo.startOffsetS

Emotion Event

// Get behavior affected by emotions.
packet.emotion.behavior.code.NEUTRAL
packet.emotion.behavior.code.DISGUST
packet.emotion.behavior.code.CONTEMPT
packet.emotion.behavior.code.BELLIGERENCE
packet.emotion.behavior.code.DOMINEERING
packet.emotion.behavior.code.CRITICISM
packet.emotion.behavior.code.ANGER
packet.emotion.behavior.code.TENSION
packet.emotion.behavior.code.TENSE_HUMOR
packet.emotion.behavior.code.DEFENSIVENESS
packet.emotion.behavior.code.WHINING
packet.emotion.behavior.code.SADNESS
packet.emotion.behavior.code.STONEWALLING
packet.emotion.behavior.code.INTEREST
packet.emotion.behavior.code.VALIDATION
packet.emotion.behavior.code.AFFECTION
packet.emotion.behavior.code.HUMOR
packet.emotion.behavior.code.SURPRISE
packet.emotion.behavior.code.JOY

// Get strength of the emotion.
packet.emotion.strength.code.UNSPECIFIED
packet.emotion.strength.code.WEAK
packet.emotion.strength.code.STRONG
packet.emotion.strength.code.NORMAL

Trigger Event

// Trigger name.
packet.trigger.name
// Parameters that come with given event.
packet.parameters

// Parameter name.
parameter.name
// Parameter value.
parameter.value

Control Event

packet.control.type.INTERACTION_END

Silence Event

// Silence duration in milliseconds.
packet.silence.durationMs

Cancel Response Event

// Interaction id for response cancellation.
packet.cancelResponses.interaction_id
// Utterance ids to cancel for the given interaction id.
packet.cancelResponses.utterance_id

Error handling

import {
  status,
} from '@inworld/nodejs-sdk';

handleError(err: ServiceError) {
    switch (err.code) {
        // Cancelled by server due timeout inactivity.
        case status.ABORTED:
        // Cancelled by client.
        case status.CANCELLED:
            break;
        default:
            console.error(err);
            break;
    }
};

You can read more about possible GRPC status codes here.

Environment

Use these environment variables for local debugging of the package.

Name	Description	Default value
NODE_SDK_INWORLD_STUDIO_HOST	Studio endpoint	api-studio.inworld.ai:443
NODE_SDK_INWORLD_STUDIO_SSL	Define if Studio connection uses TLS	true
NODE_SDK_INWORLD_ENGINE_HOST	API endpoint	api-engine.inworld.ai:443
NODE_SDK_INWORLD_ENGINE_SSL	Define if Engine connection uses TLS	true