Image, audio and PDF inputs

Send images, audio recordings or PDF documents alongside text for a model to analyze. These are chat completion inputs; to generate an image, see Image generation. Input support depends on the model. Send each input in a message's content array, with one typed part per piece of content.

Whether a model accepts each input depends on the model. To find models that do, check spec.inputModalities in the List models response: it lists image for models that take images, audio for models that take audio, and file for models that take PDF documents. For images only, spec.capabilities.vision gives the same answer. Refer to the provider's documentation for supported formats and limits.

Vision

Pass the image as an HTTP(S) URL or as a base64 data URI in image_url.url:

curl --request POST \
  --url https://api.inworld.ai/v1/chat/completions \
  --header "Authorization: Basic $INWORLD_API_KEY" \
  --header 'Content-Type: application/json' \
  --data '{
    "model": "google-ai-studio/gemini-2.5-flash",
    "messages": [
      {
        "role": "user",
        "content": [
          { "type": "text", "text": "Describe this image." },
          { "type": "image_url", "image_url": { "url": "data:image/jpeg;base64,/9j/4AAQ..." } }
        ]
      }
    ]
  }'

image_url.detail is optional and defaults to auto.

Image URLs on Google models

Google AI Studio and Vertex AI do not fetch images from HTTP(S) URLs. Send those models a base64 data URI.

The router checks this against the models a request can reach:

• If every candidate model is served by Google, an HTTP(S) image URL returns 400.
• If the request can also reach a model from another provider, for example through fallbacks or model: "auto", the router uses that model instead.

A data URI works on every provider, so prefer it when a request can route to more than one.

Audio input

Pass base64-encoded audio in input_audio.data and name its format:

curl --request POST \
  --url https://api.inworld.ai/v1/chat/completions \
  --header "Authorization: Basic $INWORLD_API_KEY" \
  --header 'Content-Type: application/json' \
  --data '{
    "model": "google-ai-studio/gemini-2.5-flash",
    "messages": [
      {
        "role": "user",
        "content": [
          { "type": "text", "text": "Transcribe this audio." },
          { "type": "input_audio", "input_audio": { "data": "UklGRi...", "format": "wav" } }
        ]
      }
    ]
  }'

The router passes format to the provider without checking it, so the formats you can use are the ones the model accepts. Anthropic models do not take audio input: the audio part is left out of the request and the model sees only the rest of the message.

PDF input

Pass a PDF as a base64 data URI in file.file_data. filename is optional. Read the file and encode it first. These examples send report.pdf from the current directory.

# Base64-encode the PDF on one line, then build the request body from it.
base64 < report.pdf | tr -d '\n' > report.pdf.b64
jq -n --rawfile pdf report.pdf.b64 '{
  model: "google-ai-studio/gemini-2.5-flash",
  messages: [{
    role: "user",
    content: [
      { type: "text", text: "Summarize this report." },
      { type: "file", file: { file_data: ("data:application/pdf;base64," + $pdf), filename: "report.pdf" } }
    ]
  }]
}' > request.json

curl --request POST \
  --url https://api.inworld.ai/v1/chat/completions \
  --header "Authorization: Basic $INWORLD_API_KEY" \
  --header 'Content-Type: application/json' \
  --data @request.json

import base64
import os

from openai import OpenAI

api_key = os.environ["INWORLD_API_KEY"]
client = OpenAI(
    base_url="https://api.inworld.ai/v1",
    api_key=api_key,
    default_headers={"Authorization": f"Basic {api_key}"},
)

with open("report.pdf", "rb") as f:
    pdf_base64 = base64.b64encode(f.read()).decode("ascii")

response = client.chat.completions.create(
    model="google-ai-studio/gemini-2.5-flash",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Summarize this report."},
                {
                    "type": "file",
                    "file": {
                        "file_data": f"data:application/pdf;base64,{pdf_base64}",
                        "filename": "report.pdf",
                    },
                },
            ],
        }
    ],
)
print(response.choices[0].message.content)

The cURL version writes the body to a file and sends it with --data @request.json, because a PDF encoded on the command line can exceed the shell's argument limit.

The router sends the PDF to the model as you sent it, and the model reads the document itself. Use a model whose spec.inputModalities lists file.

• Send the PDF as a data URI with the application/pdf media type. A file part that carries another kind of file, such as an image or plain text, returns 400. Send images in an image_url part instead.
• A file_url link returns 400 on chat completions. Download the file and send its bytes in file_data.

Validation

• A data URI in image_url must have an image/* media type, and one in input_audio an audio/* media type. Anything else returns 400.
• Video input is not supported: a video data URI in image_url returns 400.
• Content parts of any other type, such as video_url, are skipped rather than rejected. The request can succeed without the model having seen that part, so check that every part you send is one of the four types above.

How routing uses input types

• With model: "auto", the router only considers models that accept the input types in your request.
• With a named model, that model is always tried, and only its fallbacks are filtered by input type.

Related

• Voice responses return spoken audio from a chat completion.
• Chat completions covers the request basics.
• API reference lists every parameter.