Video generation

Reference-to-video

Guide a video with reference images, videos and audio clips that show the model subjects, motion and sound

References show the Video API what to include without fixing any frame of the video. Send them in input_references on the same POST /v1/videos request, then poll and download the job as usual. The prompt decides what happens; the references decide what the subjects look and sound like.

Reference type	Item	Use it to
Image	`image_url`	Keep a character, product or object consistent across videos
Video	`video_url`	Show the model a subject in motion
Audio	`audio_url`	Give the model a sound to work from, such as a voice

A request uses either references or frame_images, not both. To start from a first frame or end on a last frame, see Image-to-video.

Choose a model

Browse inworld.ai/models and select Video. Models differ in which reference types they accept and how many. See Model compatibility for the documented models.

The steps below reuse the INWORLD_API_KEY and VIDEO_REQUEST_KEY variables from Generate your first video. Use a new VIDEO_REQUEST_KEY for each new video. The Python examples need the requests package (python -m pip install requests). The Node.js examples use top-level await, so save them as .mjs files and run them with Node.js 20 or later.

Prepare your references

Upload each file, then replace the example URLs on this page with your own. Each URL must be HTTPS and download without your browser session or additional authentication headers.

URL rules checked at creation

The API rejects the create request with 400 if a reference URL breaks these rules:

It must use HTTPS and name a public host. HTTP URLs, private-network IP addresses, and hosts such as localhost or names ending in .internal or .local are rejected.
It can be at most 4,096 bytes long.
It must be a link, not inline data. Base64 data: URLs are rejected. Upload the file somewhere the provider can download it, such as your own storage bucket.

Video references

The API downloads each video reference when you create the job, to check it before anything is generated. A video reference that fails these checks is rejected with 400:

It must be an MP4 or MOV file of up to 100 MB.
All the request's video references must download within 60 seconds in total.
Each clip and the clips' total length must fit the model's limits in Model compatibility.

Because of these downloads, a create request with video references can take up to a minute. Set your HTTP client's timeout to at least 90 seconds, so it doesn't give up while the API is still checking the files.

Audio references

The API doesn't download audio references, so it doesn't check their format or length when you create the job. If the model can't use an audio clip, the job fails after it was created.

Recommendations

These aren't checked when you create the job. If the provider can't download or use a reference, the job fails after it was created:

Keep signed URLs valid for at least an hour. The provider downloads each reference itself when generation starts, after you create the job. The API's check of a video reference at creation doesn't keep a copy for the provider. One hour is a recommendation, not an API limit.
Check each URL before you send it. Confirm it opens and returns the right kind of file.

The model provider downloads each reference directly from the URL you send, so a signed URL, including its signature, is shared with that provider. Use short-lived, read-only URLs.

Use reference images

Put images of the subjects in input_references:

bash

curl -X POST https://api.inworld.ai/v1/videos \
  -H "Authorization: Basic $INWORLD_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $VIDEO_REQUEST_KEY" \
  -d '{
    "model": "alibaba/wan-3.0",
    "prompt": "The woman in the red coat walks into a busy cafe and sits by the window",
    "seconds": 5,
    "resolution": "720p",
    "aspect_ratio": "16:9",
    "input_references": [
      {
        "type": "image_url",
        "image_url": { "url": "https://storage.example.com/characters/red-coat.png" }
      }
    ]
  }'

python

import os

import requests

response = requests.post(
    "https://api.inworld.ai/v1/videos",
    headers={
        "Authorization": f"Basic {os.environ['INWORLD_API_KEY']}",
        # Resend with the same key after a timeout: you get this job back, not a second one.
        "Idempotency-Key": os.environ["VIDEO_REQUEST_KEY"],
    },
    json={
        "model": "alibaba/wan-3.0",
        "prompt": "The woman in the red coat walks into a busy cafe and sits by the window",
        "seconds": 5,
        "resolution": "720p",
        "aspect_ratio": "16:9",
        "input_references": [
            {
                "type": "image_url",
                "image_url": {
                    "url": "https://storage.example.com/characters/red-coat.png",
                },
            },
        ],
    },
    timeout=30,
)
response.raise_for_status()
video = response.json()
print(video["id"], video["status"], video["aspect_ratio"])

javascript

const response = await fetch('https://api.inworld.ai/v1/videos', {
  method: 'POST',
  headers: {
    Authorization: `Basic ${process.env.INWORLD_API_KEY}`,
    'Content-Type': 'application/json',
    // Resend with the same key after a timeout: you get this job back, not a second one.
    'Idempotency-Key': process.env.VIDEO_REQUEST_KEY,
  },
  body: JSON.stringify({
    model: 'alibaba/wan-3.0',
    prompt: 'The woman in the red coat walks into a busy cafe and sits by the window',
    seconds: 5,
    resolution: '720p',
    aspect_ratio: '16:9',
    input_references: [
      {
        type: 'image_url',
        image_url: {
          url: 'https://storage.example.com/characters/red-coat.png',
        },
      },
    ],
  }),
});
if (!response.ok) throw new Error(`${response.status}: ${await response.text()}`);
const video = await response.json();
console.log(video.id, video.status, video.aspect_ratio);

aspect_ratio works as it does for text-only videos.
The number of references a model accepts is in Model compatibility. Sending more is rejected with 400.

Add video and audio references

Add video_url and audio_url items to input_references, alongside or instead of images. This request uses an image, a video and an audio clip, and waits up to 90 seconds for the create call:

bash

curl -X POST https://api.inworld.ai/v1/videos \
  --max-time 90 \
  -H "Authorization: Basic $INWORLD_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $VIDEO_REQUEST_KEY" \
  -d '{
    "model": "alibaba/wan-3.0",
    "prompt": "The woman in Image 1 walks like the dancer in Video 1, speaking in the voice from Audio 1",
    "seconds": 5,
    "resolution": "720p",
    "aspect_ratio": "16:9",
    "input_references": [
      {
        "type": "image_url",
        "image_url": {
          "url": "https://storage.example.com/characters/red-coat.png"
        }
      },
      {
        "type": "video_url",
        "video_url": {
          "url": "https://storage.example.com/motion/dance.mp4"
        }
      },
      {
        "type": "audio_url",
        "audio_url": {
          "url": "https://storage.example.com/voices/narrator.wav"
        }
      }
    ]
  }'

python

import os

import requests

response = requests.post(
    "https://api.inworld.ai/v1/videos",
    headers={
        "Authorization": f"Basic {os.environ['INWORLD_API_KEY']}",
        # Resend with the same key after a timeout: you get this job back, not a second one.
        "Idempotency-Key": os.environ["VIDEO_REQUEST_KEY"],
    },
    json={
        "model": "alibaba/wan-3.0",
        "prompt": "The woman in Image 1 walks like the dancer in Video 1, speaking in the voice from Audio 1",
        "seconds": 5,
        "resolution": "720p",
        "aspect_ratio": "16:9",
        "input_references": [
            {
                "type": "image_url",
                "image_url": {
                    "url": "https://storage.example.com/characters/red-coat.png",
                },
            },
            {
                "type": "video_url",
                "video_url": {
                    "url": "https://storage.example.com/motion/dance.mp4",
                },
            },
            {
                "type": "audio_url",
                "audio_url": {
                    "url": "https://storage.example.com/voices/narrator.wav",
                },
            },
        ],
    },
    # The API downloads video references while it creates the job.
    timeout=90,
)
response.raise_for_status()
video = response.json()
print(video["id"], video["status"])

javascript

const response = await fetch('https://api.inworld.ai/v1/videos', {
  method: 'POST',
  headers: {
    Authorization: `Basic ${process.env.INWORLD_API_KEY}`,
    'Content-Type': 'application/json',
    // Resend with the same key after a timeout: you get this job back, not a second one.
    'Idempotency-Key': process.env.VIDEO_REQUEST_KEY,
  },
  body: JSON.stringify({
    model: 'alibaba/wan-3.0',
    prompt: 'The woman in Image 1 walks like the dancer in Video 1, speaking in the voice from Audio 1',
    seconds: 5,
    resolution: '720p',
    aspect_ratio: '16:9',
    input_references: [
      {
        type: 'image_url',
        image_url: {
          url: 'https://storage.example.com/characters/red-coat.png',
        },
      },
      {
        type: 'video_url',
        video_url: {
          url: 'https://storage.example.com/motion/dance.mp4',
        },
      },
      {
        type: 'audio_url',
        audio_url: {
          url: 'https://storage.example.com/voices/narrator.wav',
        },
      },
    ],
  }),
  // The API downloads video references while it creates the job.
  signal: AbortSignal.timeout(90_000),
});
if (!response.ok) throw new Error(`${response.status}: ${await response.text()}`);
const video = await response.json();
console.log(video.id, video.status);

Replace the example URLs with your own files. To poll the job and download the video, use the scripts in Video generation with this request body and a 90-second create timeout.

You can send audio references on their own, without an image or video reference.
Reference videos add to the cost of a job; see Billing.
Not every model accepts video or audio references. Sending a type the model doesn't accept is rejected with 400.

Refer to more than one reference

The API passes references to the model in the order you send them. In your prompt, name each reference by its type and position, and describe what each shows so the model can tell the subjects apart. Each type is numbered separately: the first image is Image 1, the first video is Video 1, and the first audio clip is Audio 1.

json

"prompt": "The woman in Image 1 carries the teapot in Image 2 into a sunlit kitchen",
"input_references": [
  {
    "type": "image_url",
    "image_url": { "url": "https://storage.example.com/characters/red-coat.png" }
  },
  {
    "type": "image_url",
    "image_url": { "url": "https://storage.example.com/products/teapot.png" }
  }
]

Errors

References are validated when you create the job. A rejected request returns 400 with code: "invalid_request" and a param that points to the item at fault. The message never repeats the URL.

`param`	Cause
`input_references[0].type`	The item's `type` isn't `image_url`, `video_url` or `audio_url`
`input_references[0].image_url.url`, `input_references[0].video_url.url`, `input_references[0].audio_url.url`	The URL is missing, isn't HTTPS, names a private host, or is longer than 4,096 bytes
`input_references[0].video_url`	The video couldn't be downloaded, is larger than 100 MB, isn't a readable MP4 or MOV file, or its length is outside the model's limits
`input_references`	Too many references of a type, a type the model doesn't accept, video references longer than the model's total, or combined with `frame_images`

If the job fails after creation

A valid URL doesn't guarantee a usable reference. The job can still fail after it was created, with status: "failed". To recover:

Read error.code on the job. A reference problem usually shows up as invalid_request (the provider rejected the request or the file) or generation_failed, and content the provider's content policy blocks as content_filtered. See Job failures.
Check that each URL still downloads without a browser session or extra headers, and that it returns the right kind of file.
Submit the corrected request with a new Idempotency-Key. Resending the old key returns the failed job.

Model compatibility

Reference support by model

The table below describes reference support for the models shown; it is not an exhaustive catalog:

	Wan 3.0 (`alibaba/wan-3.0`)	MiniMax H3 (`minimax/minimax-h3`)	Grok Imagine Video 1.5 Lite (`xai/grok-imagine-video-1.5-lite`)
Images	Up to 10	Up to 9	No
Videos	Up to 5	Up to 3	No
Audio clips	Up to 5	Up to 3	No
All references together	No separate limit	Up to 12	—
Each video's length	1 to 15 seconds	2 to 15 seconds	—
All videos' length together	Up to 15 seconds	Up to 15 seconds	—