> ## Documentation Index
> Fetch the complete documentation index at: https://docs.inworld.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Use an LLM

The `node-llm-chat` template illustrates how to make LLM calls using the LLM node with support for streaming, tool calling, and multimodal inputs.

<Note>
  **Architecture**

  * **Backend:** Inworld Agent Runtime
  * **Frontend:** N/A (CLI example)
</Note>

## Prerequisites

* Node.js v20 or higher: [Download here](https://nodejs.org/en/download)
* Inworld API key (required): [Sign up here](https://platform.inworld.ai/signup) or see [quickstart guide](/node/authentication#getting-an-api-key)

## Run the Template

1. Clone the [templates repository](https://github.com/inworld-ai/inworld-runtime-templates-node):
   ```bash theme={"system"}
   git clone https://github.com/inworld-ai/inworld-runtime-templates-node
   cd inworld-runtime-templates-node
   ```

2. Install the Runtime SDK inside the `cli` directory.

   <CodeGroup>
     ```shell Yarn theme={"system"}
     yarn add @inworld/runtime
     ```

     ```shell npm theme={"system"}
     npm install @inworld/runtime
     ```
   </CodeGroup>

3. Set up your Base64 [Runtime API key](/node/authentication) by copying the `.env-sample` file into a `.env` file in the `cli` folder and adding your API key.
   ```env .env theme={"system"}
   # Inworld Agent Runtime Base64 API key
   INWORLD_API_KEY=<your_api_key_here>
   ```

4. Run a basic example of calling the LLM with a text prompt:
   ```bash theme={"system"}
   yarn node-llm-chat "Hello, how are you?" \
     --modelName=gemini-2.5-flash --provider=google
   ```

5. Now try changing the model and requiring JSON outputs. See [Models > Chat Completion](/models#chat-completion) for models supported.
   ```bash theme={"system"}
   yarn node-llm-chat "What is the weather in Vancouver? Return in JSON format" \
     --modelName=gpt-4o --provider=openai --responseFormat=json
   ```

6. Now let's try with tool calling.
   ```bash theme={"system"}
   yarn node-llm-chat "What is 15 + 27?" \
     --modelName=gpt-4o --provider=openai --tools --toolChoice=auto
   ```

7. Now let's try with image inputs:
   ```bash theme={"system"}
   yarn node-llm-chat "What do you see in this image?" \
     --modelName=gpt-4o --provider=openai \
     --imageUrl="https://upload.wikimedia.org/wikipedia/en/a/a9/Example.jpg"
   ```

8. Finally, check out your captured traces in [Portal](https://platform.inworld.ai/)!

## Understanding the Template

The main functionality of the template is contained in the run function, which demonstrates how to use the Inworld Agent Runtime to generate text using the LLM node.

Let's break it down into more detail:

### 1) Initialize LLM node

First, we initialize the LLM node

```javascript theme={"system"}
import {
  GraphBuilder,
  GraphTypes,
  RemoteLLMChatNode,
} from '@inworld/runtime/graph';

const llmNode = new RemoteLLMChatNode({
  stream,
  provider,
  modelName,
});
```

When creating the node, you can specify:

* **provider**: The LLM service provider (inworld, openai, etc.) as specified [here](/models#chat-completion)
* **modelName**: Any model from [Chat Completion](/models#chat-completion)
* **stream**: Whether to enable streaming responses
* **textGenerationConfig**: LLM generation parameters. You can learn more about these configurations [here](/node/runtime-reference/interfaces/graph_dsl_graph_config_schema.TextGenerationConfig).

For convenience, `createRemoteLLMChatNode` does not require registering the component explicitly, before use in the node, but you can also register the component and reference it when creating the node as shown in the `node_llm_chat_explicit_components.ts` example. This allows you to reuse the same component across multiple nodes (for example, if you want to make multiple LLM calls with the same model).

```javascript theme={"system"}
  const llmComponent = new RemoteLLMComponent({
    provider,
    modelName,
    defaultConfig: TEXT_CONFIG_SDK,
  });

  const llmNode = new RemoteLLMChatNode({
    llmComponent,
    stream,
  });
```

### 2) Graph initialization

Next we create a new graph and add the LLM node, setting it as the start and end node. In more complex applications, you could connect multiple LLM nodes to create a processing pipeline.

```javascript theme={"system"}
const graph = new GraphBuilder({
  id: 'node_llm_chat_graph',
  enableRemoteConfig: false,
  apiKey,
})
  .addNode(llmNode)
  .setStartNode(llmNode)
  .setEndNode(llmNode)
  .build();
```

The [GraphBuilder](/node/runtime-reference/classes/graph_dsl_graph_builder.GraphBuilder) configuration includes:

* **id**: A unique identifier for the graph
* **enableRemoteConfig**: Whether to enable remote configuration (set to false for local execution)
* **apiKey**: Your Inworld API key

### 3) Graph input preparation

Now we create the message to send to the LLM based on the user's input.

```javascript theme={"system"}
let graphInput;

if (tools) {
  graphInput = createMessagesWithTools(
    prompt,
    toolChoice,
    imageUrl,
    toolCallHistory,
  );
} else {
  graphInput = createMessages(prompt, imageUrl, toolCallHistory);
}

if (responseFormat) {
  graphInput.responseFormat = responseFormat;
}
```

Below is an example of what different inputs might look like:

```javascript theme={"system"}
// Basic text message
const basicInput = {
  messages: [
    {
      role: 'system',
      content: 'You are a helpful assistant that can use tools when needed. When analyzing images, describe what you see and use appropriate tools if calculations or weather information is needed.'
    },
    {
      role: 'user', 
      content: prompt
    }
  ]
};

// Multimodal message with image
const multimodalInput = {
  messages: [
    {
      role: 'user',
      content: [
        {
          type: 'text',
          text: prompt,
        },
        {
          type: 'image',
          image_url: {
            url: imageUrl,
            detail: 'high',
          },
        },
      ],
    }
  ]
};

// Messages with tool support
const toolInput = {
  messages: [...],
  tools: [
    {
      name: 'calculator',
      description: 'Evaluate a mathematical expression',
      properties: {
        type: 'object',
        properties: {
          expression: {
            type: 'string',
            description: 'The mathematical expression to evaluate',
          },
        },
        required: ['expression'],
      },
    },
    {
      name: 'get_weather',
      description: 'Get the current weather in a location',
      properties: {
        type: 'object',
        properties: {
          location: {
            type: 'string',
            description: 'The city and state, e.g., San Francisco, CA',
          },
        },
        required: ['location'],
      },
    },
  ],
  toolChoice: {
    choice: 'auto' // or 'required', 'none', or specific function name
  }
};
```

### 4) Graph execution

Execute the graph with the prepared input:

```javascript theme={"system"}
const { outputStream } = await graph.start(new GraphTypes.LLMChatRequest(graphInput));
```

### 5) Response handling

Handle responses, including streaming responses.

```javascript theme={"system"}
for await (const result of outputStream) {
  await result.processResponse({
    Content: (response: GraphTypes.Content) => {
      console.log('📥 LLM Chat Response:');
      console.log('  Content:', response.content);
      
      // Handle tool calls if present
      if (response.toolCalls && response.toolCalls.length > 0) {
        console.log('  Tool Calls:');
        response.toolCalls.forEach((toolCall, index) => {
          console.log(`    ${index + 1}. ${toolCall.name}(${toolCall.args})`);
          console.log(`       ID: ${toolCall.id}`);
        });
      }
    },
    ContentStream: async (stream: GraphTypes.ContentStream) => {
      console.log('📡 LLM Chat Response Stream:');
      let streamContent = '';
      const toolCalls: { [id: string]: any } = {};
      let chunkCount = 0;
      
      for await (const chunk of stream) {
        chunkCount++;
        if (chunk.text) {
          streamContent += chunk.text;
          process.stdout.write(chunk.text);
        }
        
        // Accumulate tool calls from stream
        if (chunk.toolCalls && chunk.toolCalls.length > 0) {
          for (const toolCall of chunk.toolCalls) {
            if (toolCalls[toolCall.id]) {
              toolCalls[toolCall.id].args += toolCall.args;
            } else {
              toolCalls[toolCall.id] = { ...toolCall };
            }
          }
        }
      }
      
      console.log(`\nTotal chunks: ${chunkCount}`);
      console.log(`Final content length: ${streamContent.length} characters`);
      
      const finalToolCalls = Object.values(toolCalls);
      if (finalToolCalls.length > 0) {
        console.log('Tool Calls from Stream:');
        finalToolCalls.forEach((toolCall, index) => {
          console.log(`  ${index + 1}. ${toolCall.name}(${toolCall.args})`);
          console.log(`     ID: ${toolCall.id}`);
        });
      }
    },
    default: (data: any) => {
      console.error('Unprocessed response:', data);
    },
  });
}
```