Skip to main content

CLI examples

Find source code of examples here.

Requirements​

  • Latest version of sox installed and available in PATH

How to combine events with different types​

In most cases, a user's input can generate various types of events, including text, audio, emotions, and more. To apply multiple types of events simultaneously, such as playing audio and displaying text, you can connect them using the package.packageId.utteranceId.

Automatic reconnect​

Connection will be opened automatically on send if closed, or else an existing open connection will be used.

CLI interface​

You will need to use the following console commands to start the client with auto reconnect:

|- /start - starts audio capturing.
|- /end - ends audio capturing.
|- /trigger - send trigger.
|- /info - shows current character.
|- /list-all - shows available characters (created within the scene).
|- /character %character-id% - id of the target character (Get full list using /list-all command).
|- c - cancel current response.
|- <any_other_text> - sends text event to server.

Setup connection​

const client = new InworldClient()
  .setApiKey({
    key: process.env.INWORLD_KEY!,
    secret: process.env.INWORLD_SECRET!,
  })
  .setScene(process.env.INWORLD_SCENE!)
  .setOnError((err: ServiceError) => console.error(`Error: ${err.message}`))
  .setOnDisconnect(() => console.log('Disconnected'))
  .setOnMessage(() => console.log('Message was received'))
  .build();
}

Get list of characters and select one of them​

If the scene contains more than one character, then the first one will be used by default. You can change this manually by running:

const characters = await connection.getCharacters();
const character = characters.find((c) => c.id === id);

connection.setCurrentCharacter(character);

This should produce the following console output,

/info
Character: 4bafd052-fe3e-483b-9eaf-ab20cadf84b2 (workspaces/test-workspace/characters/character1:First character)
------------------------------
/list-all
0: 4bafd052-fe3e-483b-9eaf-ab20cadf84b2 (workspaces/test-workspace/characters/character1:First character)
1: 82825e9c-7d80-42a4-b6b2-f313c1ac29ed (workspaces/test-workspace/characters/character2:Second character)
------------------------------
/character 82825e9c-7d80-42a4-b6b2-f313c1ac29ed
Character: 82825e9c-7d80-42a4-b6b2-f313c1ac29ed (workspaces/test-workspace/characters/character2:Second character)
------------------------------

Sending a text message, trigger, or cancelling the response to a character​

The connection will be opened automatically on send if it is closed, or an existing open connection will be used. Our server will close the connection automatically in one minute if there is inactivity.

await connection.sendText('Hello');
await connection.sendTrigger('Some action');
await connection.sendCancelResponse();

Sending audio chunk to character​

Initialize recorder and send audio on each data chunk that it receives. Before beginning capture, send an audio session start event to the connection. Do not forget to send an audio session end event at the end of the capture.

await connection.sendAudioSessionStart();
// Start audio capturing here
await connection.sendAudio(chunk);
...
// Stop audio capturing here
await connection.sendAudioSessionEnd();

This should produce the following console output:

/start
Recognized: How are you?, final=true
Character(aa1c8634-df62-4d8b-94eb-4188cf8fbea7) to Player (i=599d9d2e-9994-4208-bb52-b8d5d8997819, u=620d9711-42d4-433f-b5e9-455ac0b7f6aa): I\'m good, thanks!
Character(aa1c8634-df62-4d8b-94eb-4188cf8fbea7) to Player (i=211644a2-2657-40be-b6e6-b88194604717, u=606c9cf7-334b-43e6-af4e-269b78083a87):  Thank you for asking.
/end

Close connection​

You can close the connection manually if it is not currently required. Be sure to do this on your application stop, e.g., ‘SIGINT’ event. It will be opened automatically when the next packet is sent.

connection.close();

Manual reconnect​

If the connection is closed, you will need to open it manually before sending any data.

CLI interface​

You will need to use the following console commands to start the client with auto reconnect:

|- /open - open connection.
|- /close - close connection.
|- /start - starts audio capturing.
|- /end - ends audio capturing.
|- /trigger - send trigger.
|- /info - shows current character.
|- /list-all - shows available characters (created within the scene).
|- /character %character-id% - id of the target character (Get full list using /list-all command).
|- c - cancel current response.
|- <any_other_text> - sends text event to server.

Setup connection​

If you would like to open and close the connection manually, then you should ensure the property autoReconnect = false is set in the configuration. It's also possible to specify a disconnectTimeout (1 minute is used by default).

const client = new InworldClient()
  .setApiKey({
    key: process.env.INWORLD_KEY!,
    secret: process.env.INWORLD_SECRET!,
  })
  .setScene(process.env.INWORLD_SCENE!)
  .setConfiguration({
    connection: {
      autoReconnect: false,
      disconnectTimeout: 30 * 1000, // 30 seconds
    },
  })
  .setOnError((err: ServiceError) => console.error(`Error: ${err.message}`))
  .setOnDisconnect(() => console.log('Disconnected'))
  .setOnMessage(() => console.log('Message was received'))
  .build();
}

Open connection​

You should open the connection and manage it manually as follows,

  if (!connection.isActive()) {
    await connection.open();
  }
  // Do something here

Send message to character​

If the connection is open, a message will be sent to the character. Otherwise, you will receive an error message.

await connection.sendText('Hello');

Close connection​

You can close the connection manually if it is not currently required. Be sure to do this on your application stop, e.g., ‘SIGINT’ event. It can be opened later.

connection.close();

Interruptions​

It's possible for the character to be interrupted during a conversation. To enable this feature, you need to add the interruptions capability to the character configuration."

CLI interface​

You will need to use the following console commands to start the client with interruptions:

|- /start - starts audio capturing.
|- /end - ends audio capturing.
|- /info - shows current character.
|- /list-all - shows available characters (created within the scene).
|- /character %character-id% - id of the target character (Get full list using /list-all command).

Setup connection​

Don't forget to set up the interruptions capability.

const client = new InworldClient()
  .setApiKey({
    key: process.env.INWORLD_KEY!,
    secret: process.env.INWORLD_SECRET!,
  })
  .setConfiguration({
    capabilities: { interruptions: true },
  })
  .setScene(process.env.INWORLD_SCENE!)
  .setOnError((err: ServiceError) => console.error(`Error: ${err.message}`))
  .setOnDisconnect(() => console.log('Disconnected'))
  .setOnMessage(() => console.log('Message was received'))
  .build();
}

Messages management​

During a character's speech (i.e., when audio packets are playing), the user can interrupt the character. This occurs in the following situation: when the user has said something and the text has been recognized (you can use packet.routing.source.isPlayer to detect it).

  • When the recognition event is received, ensure if the character is currently speaking by checking the audio packet queue for the same interactionId.
  • If the character is speaking, remove all audio and text events with the same interactionId from the queue.
  • Call the sendCancelResponse method with the utteranceId and interactionId to cancel any events that were not applied on the server-side.
  • If the character is not currently speaking, do nothing and continue playing the audio packets.

Please refer to the provided source code for implementation details.

Handlers​

onDisconnect​

Each time the server closes the connection or you call connection.close, a disconnect event will be triggered.

connection.setOnDisconnect(() => console.log('Disconnected'));

onError​

connection.setOnError((err: ServiceError) => console.error(`Error: ${err.message}`));

onMessage​

To detect the type of message and print or play, you can use,

connection.setOnMessage(async (packet: InworldPacket) => {
  const { packetId } = packet;
  const i = packetId.packetId;
  const u = packetId.utteranceId;

  // TEXT
  if (packet.isText()) {
    const textEvent = packet.text;

    if (packet.routing.source.isPlayer) {
      if (textEvent.final) {
        console.log(
          `Recognized: ${textEvent.text}, final=${textEvent.final}`,
        );
      }
    } else {
      console.log(
        `${this.renderEventRouting(packet)} (i=${i}, u=${u}): ${
          textEvent.text
        }`,
      );
    }
  }

  // AUDIO
  if (packet.isAudio()) {
    player.send(packet.audio.chunk);
  }
});