OpenClaw video generation

OpenClaw agents generate videos from text prompts, reference images or existing videos through one tool, with sixteen provider backends behind it and the right one picked automatically from config and available API keys. The docs put the gate first: the tool only appears when at least one video-generation provider is available. Here are the modes, the asynchronous lifecycle, the provider table, the parameters, the selection and fallback rules, and the provider notes that decide what works where.

Modes and quick start

• Three runtime modes resolved from the reference inputs: generate with no reference media, image-to-video with one or more reference images, video-to-video with one or more reference videos; the tool validates the active mode before submission and the list action reports what each provider supports.
• Quick start: set an API key for any supported provider, optionally pick a default model in the media models config, and ask the agent; no tool allowlisting is needed.
• Generation is asynchronous: the request returns a task id, the provider works for thirty seconds to several minutes, and OpenClaw wakes the same session with a completion event so the agent reports the result through its normal reply path, with an idempotent direct fallback if the session is inactive.
• Duplicate calls during a job return the current status instead of starting another, a status action checks without generating, the tasks CLI lists, shows and cancels, and outside a session-backed run the tool falls back to inline generation.

video_generate only appears when at least one video-generation provider is available.

Providers and parameters

The table runs from Alibaba and Qwen on DashScope, which need remote URLs for references, through three BytePlus Seedance generations with first and last frame roles and, on the 2.0 line, up to nine images, three videos and three audio references, ComfyUI workflows, DeepInfra text-to-video, fal with a queue that waits up to twenty minutes, Google Veo with one image or one video, MiniMax with its two resolutions, OpenAI Sora with only the size override forwarded, OpenRouter’s asynchronous videos API, Runway with data-URI local files and a video-to-video model of its own, Together, Vydra, and xAI’s Grok video models with a 1.5 line that is image-to-video only. Parameters: the prompt; one image or up to nine with per-position roles, first frame, last frame or reference image; one video or up to four; up to three reference audios; aspect ratio, resolution and duration hints that are normalised or ignored per provider, a size hint, generated audio and watermark toggles; an action of generate, status or list; a model override; a filename hint; a timeout; and typed provider options. Generated files save under managed media storage with a sixteen-megabyte default cap, and a hosted URL is delivered instead if a larger render cannot be persisted.

Selection and fallback

• Model selection order: the model parameter alone when set, then the configured primary, then the fallbacks in order, then auto-detection from configured provider defaults only when nothing is configured, the default provider first and the rest alphabetically; an explicit list limits fallback to that list.
• Capability checks apply at the fallback layer, so a request past the primary’s limits can still run on a capable fallback: a candidate without audio-reference support is skipped when audio is supplied, the same for image and video counts, a duration above the candidate’s maximum skips it, and provider options that miss a typed schema skip it too; the first skip logs a warning, later ones stay quiet, and if every candidate is skipped the error lists each reason.
• Reference audio never changes the mode, it rides on top of the image or video choice, and mixing image and video references is not a stable shared capability, so the docs say to prefer one reference type per request.

OpenClaw image generation is the still-image sibling of this tool, and OpenClaw automations explains the background task machinery the async lifecycle uses.

Provider notes that matter

A few notes decide real outcomes: Google ignores generated-audio requests with a warning on the Gemini path, MiniMax normalises a 720p request to its closest supported resolution, OpenAI ignores aspect ratio, resolution, audio and watermark overrides, Runway needs its Aleph model for video-to-video, Vydra’s default is text-only while its other model needs a remote image URL, and xAI defaults to 480p, inherits the source ratio for single-image runs, and accepts two to ten seconds for extensions. OpenClaw models and providers covers the provider keys the auto-detection walks, and OpenClaw and Gemini the provider whose key is the docs’ quick-start example.

On Diali

On Diali the video tool appears once you add a provider key in the vault; the render lands in the assistant’s media storage and comes back in the chat as a file. Hosted OpenClaw on Diali is the assistant.

• No provider key, no tool.
• Text, image or video reference picks the mode.
• The job is async; the session is woken when it lands.