API
Capabilities configuration
const capabilities = new Capabilities({ emotions: true });
Name | Description | Default value |
---|---|---|
audio | If false , then the client will not receive spoken audio from the characters (text only mode). | true |
emotions | You will receive character emotions. | false |
interruptions | Allow interruptions to conversations with the character. | false |
phonemes | Include phoneme information in Audio Events. | false |
silence | Allow for pauses between character replies. | false |
Connection configuration
const connection = {
disconnectTimeout: 10 * 1000, // time in milliseconds
autoReconnect: false, // true by default
}
Name | Description | Default value |
---|---|---|
disconnectTimeout | Close connection due to inactivity | 60000 |
autoReconnect | Connection will be opened automatically on send if it is closed. Otherwise, an existing open connection will be used. Our server closes the connection automatically after 1 minute of inactivity. | true |
SessionToken
const token = client.generateSessionToken();
// Get a token.
token.token
// Get type (Bearer).
token.type
// Get expiration time.
// Token should be regenerated after expiration time.
token.expirationTime
// Get session id.
token.sessionId
InworldClient with API key
If you prefer to use automatic session management set client key/secret.
const client = new InworldClient();
// ApiKey is required.
client.setApiKey({ key, secret });
// User is not required.
client.setUser(user);
// Configuration is not required.
client.setConfiguration({
capabilities,
connection: {
disconnectTimeout: 10 * 1000, // time in milliseconds
autoReconnect: false,
}
});
// Scene is not required for token generation.
// But if you would like to speak with scene characters you need to provide scene full name.
// It should be like workspaces/{WORKSPACE_NAME}/characters/{CHARACTER_NAME}.
// Or like workspaces/{WORKSPACE_NAME}/scenes/{SCENE_NAME}.
client.setScene(scene);
// Event handlers
client.setOnDisconnect(fn);
client.setOnError(fn);
client.setOnMessage(fn);
// Generate SessionAccessToken.
client.generateSessionToken();
// Finish connection configuration.
// Return instance of EventService.
// Сonnection is not open. It's just ready to open on message send.
const connection = client.build();
Manage session manually
If you prefer to manage a session manually you should provide a onSessionGet
and onSessionSet
functions. This will allow you to store session
externally (for example, in redis) and reuse a session on application restart. See the Discord bot example for more details.
import { Session } from '@inworld/nodejs-sdk';
const key = 'character_key';
async function onSessionGet() {
const json = await redisClient.get(key);
return Session.deserialize(json);
}
function onSessionSet(key: string, entity: Session) {
redisClient.set(key, Session.serialize(entity));
}
const client = new InworldClient();
// User is not required.
client.setUser(user);
// Configuration is not required.
client.setConfiguration({
capabilities,
connection: {
disconnectTimeout: 10 * 1000, // time in milliseconds
autoReconnect: false,
}
});
// It should be like workspaces/{WORKSPACE_NAME}/characters/{CHARACTER_NAME}.
// Or like workspaces/{WORKSPACE_NAME}/scenes/{SCENE_NAME}.
client.setScene(scene);
// Event handlers
client.setOnDisconnect(fn);
client.setOnError(fn);
client.setOnMessage(fn);
// Allow session data to be kept in some storage, such as Redis, and reused.
// This reduces the number of API calls if the session has not expired.
client.setOnSession({
get: onSessionGet,
set: onSessionSet,
});
// Finish connection configuration.
// Return instance of EventService.
// Сonnection is not open. It's just ready to open on message send.
const connection = client.build();
InworldConnectionService
const connection = client.build();
// Open connection manually. Available only if configuration.connection.autoReconnect = false.
// Otherwise connection will be managed automatically by SDK.
connection.open();
// You can check if the connection is open or not in case of configuration.connection.autoReconnect = false.
connection.isActive();
// Send text message.
connection.sendText(message: string);
// Send trigger. Pass trigger name and parameters as arguments.
interface TriggerParameter {
name: string;
value: string;
}
connection.sendTrigger(name: string, parameters?: TriggerParameter[]);
// Send audio start event before call sendAudio.
connection.sendAudioSessionStart();
// Send audio end event after all audio chunks you would like to send.
connection.sendAudioSessionEnd();
// Send audio. Pass string chunk as argument.
connection.sendAudio(chunk);
// Send a cancel response.
// InteractionId or utteranceId can be omitted.
// When interactionId is empty, everything in the session will be removed.
// When only the utteranceId is provided, nothing happens.
// When only the interactionId is provided, everything until this interaction will be removed.
// When both the interactionId and utteranceId are provided, everything until this interaction will be removed, and utterances in this interaction will also be removed.
connection.sendCancelResponse({
interactionId?: string,
utteranceId?: string[],
});
// Close connection.
connection.close();
// Get a character list.
connection.getCharacters();
// Get the current character.
connection.getCurrentCharacter();
// Change character in the scene.
connection.setCurrentCharacter(character);
Character
const character = connection.getCurrentCharacter();
// Character id.
character.id
// Character resource name.
character.resourceName
// Character display name.
character.displayName
// Character assets.
character.assets.avatarImg
character.assets.avatarImgOriginal
character.assets.rpmModelUri
character.assets.rpmImageUriPortrait
character.assets.rpmImageUriPosture
InworldPacket
client.setOnMessage((packet: InworldPacket) => {...});
// It's a text event.
packet.isText();
// It's an audio event.
packet.isAudio();
// It's a trigger event.
packet.isTrigger();
// It's an emotion event.
packet.isEmotion();
// It's a control event.
packet.isControl();
// It's a silence event.
packet.isSilence();
// It's a cancel response event.
packet.isCancelResponse();
// It's a special control event. It means that interaction ends.
packet.isInteractionEnd();
Common Event Data
// ISO string.
packet.date
// A token that uniquely identifies the packet.
packet.packetId.packetId
// A token that uniquely identifies and groups utterances in the replies.
// Different packets may belong to the same utterance. E.g. Audio Event and Text Event of the same spoken utterance.
packet.packetId.utteranceId
// A token that uniquely identifies interaction between actors.
packet.packetId.interactionId
// Determines who sends the packet: player or character.
packet.routing.source.name
packet.routing.source.isPlayer
packet.routing.source.isCharacter
// Determines who receives the packet: player or character.
packet.routing.target.name
packet.routing.target.isPlayer
packet.routing.target.isCharacter
Text Event
// Get message of text event.
packet.text.text
// If this is the final version of the text or not.
// For instance speech recognition may take some time to finalize the text.
packet.text.final
Audio Event
// Get chunk of audio event.
packet.audio.chunk
// Get list if phonemes.
packet.audio.additionalPhonemeInfo = [];
// Get phoneme data.
// Synthesized phoneme.
phonemeInfo.phoneme
// Offset from the beginning of audio segment (in seconds).
phonemeInfo.startOffsetS
Emotion Event
// Get behavior affected by emotions.
packet.emotion.behavior.code.NEUTRAL
packet.emotion.behavior.code.DISGUST
packet.emotion.behavior.code.CONTEMPT
packet.emotion.behavior.code.BELLIGERENCE
packet.emotion.behavior.code.DOMINEERING
packet.emotion.behavior.code.CRITICISM
packet.emotion.behavior.code.ANGER
packet.emotion.behavior.code.TENSION
packet.emotion.behavior.code.TENSE_HUMOR
packet.emotion.behavior.code.DEFENSIVENESS
packet.emotion.behavior.code.WHINING
packet.emotion.behavior.code.SADNESS
packet.emotion.behavior.code.STONEWALLING
packet.emotion.behavior.code.INTEREST
packet.emotion.behavior.code.VALIDATION
packet.emotion.behavior.code.AFFECTION
packet.emotion.behavior.code.HUMOR
packet.emotion.behavior.code.SURPRISE
packet.emotion.behavior.code.JOY
// Get strength of the emotion.
packet.emotion.strength.code.UNSPECIFIED
packet.emotion.strength.code.WEAK
packet.emotion.strength.code.STRONG
packet.emotion.strength.code.NORMAL
Trigger Event
// Trigger name.
packet.trigger.name
// Parameters that come with given event.
packet.parameters
// Parameter name.
parameter.name
// Parameter value.
parameter.value
Control Event
packet.control.type.INTERACTION_END
Silence Event
// Silence duration in milliseconds.
packet.silence.durationMs
Cancel Response Event
// Interaction id for response cancellation.
packet.cancelResponses.interaction_id
// Utterance ids to cancel for the given interaction id.
packet.cancelResponses.utterance_id
Error handling
import {
status,
} from '@inworld/nodejs-sdk';
handleError(err: ServiceError) {
switch (err.code) {
// Cancelled by server due timeout inactivity.
case status.ABORTED:
// Cancelled by client.
case status.CANCELLED:
break;
default:
console.error(err);
break;
}
};
You can read more about possible GRPC status codes here.
Environment
Use these environment variables for local debugging of the package.
Name | Description | Default value |
---|---|---|
NODE_SDK_INWORLD_STUDIO_HOST | Studio endpoint | api-studio.inworld.ai:443 |
NODE_SDK_INWORLD_STUDIO_SSL | Define if Studio connection uses TLS | true |
NODE_SDK_INWORLD_ENGINE_HOST | API endpoint | api-engine.inworld.ai:443 |
NODE_SDK_INWORLD_ENGINE_SSL | Define if Engine connection uses TLS | true |