Reading what arrives

A photograph, a voice note and a thirty second clip all arrive at an assistant as bytes, and bytes are a poor thing to route on. OpenClaw can summarise inbound media before the reply pipeline runs, so that command parsing and routing work off a short piece of text instead. The original media still reaches the model as usual, which makes the whole stage optional in a useful way. What follows is how the selection, the fallbacks and the limits actually work.

What understanding produces

• For each enabled capability the pipeline collects ordered inbound media facts, selects attachments according to the attachment policy, picks the first eligible model entry by size, capability and available authentication, and moves to the next entry whenever one errors or times out.
• On success the message body becomes an image, audio or video block, audio additionally sets a transcript value, and any caption is preserved inside the block, with command parsing preferring the caption text and falling back to the transcript.
• The shipped limits are ten megabytes for images, twenty for audio and fifty for video, with a five hundred character output cap on image and video summaries, sixty second timeouts for image and audio, one hundred and twenty seconds for video, and two capability runs at a time.
• By default only the first matching attachment is processed, and switching to processing all of them labels the outputs in sequence so the model can tell one from another.

Original media is always delivered to the model as usual; when understanding fails or is disabled, the reply flow continues unchanged.

How a model is chosen

With nothing configured, OpenClaw works through a fixed order and stops at the first option that works. For images it starts with the configured image model, unless the active reply model already handles vision natively, in which case the summary block is skipped entirely and the original image goes straight to the model. Next comes the active reply model itself, when its provider supports the capability in question. For audio, configured providers are tried before any local binary, in a documented priority order that begins with Groq and OpenAI and runs through xAI, Deepgram, OpenRouter, Google and SenseAudio, Deepinfra and ElevenLabs, and finally Mistral. Only then do local command line tools come into play, and their order depends on what the process has actually observed rather than on what a build claims, so an accelerated whisper build is tried first only after Metal or CUDA has been seen in a real invocation. The processor default sits in the middle, Apple Silicon has its own entry, and the Python whisper command is the last resort, defaulting to the turbo model and downloading it automatically. Image and video capabilities have their own bundled priority lists, with Anthropic and OpenAI first for images and Google first for video. Ties are broken alphabetically by provider identifier, which keeps the behaviour predictable rather than incidental.

Behaviour worth knowing

• Audio files under one thousand and twenty four bytes are treated as empty or corrupt and skipped before transcription, and the agent receives a deterministic placeholder transcript instead of a failure.
• Provider calls for audio and video honour the standard outbound proxy environment variables, lowercase forms take precedence over uppercase, and a malformed value produces a warning and a direct fetch rather than an outage, while image understanding does not use that path at all.
• At most five skip markers are rendered per message, after which further skipped attachments collapse into a single reason neutral summary, so a pile of junk attachments cannot grow the prompt without bound.

Understanding is one stage among several that touch inbound files, and the wider picture of images, audio and camera input on a device is covered in Image and media support. Credentials follow the same resolution as ordinary model calls, so profiles, environment variables and configured provider keys all apply, and Tools and custom providers is where that is written down.

Why summarise at all

Summarising before the reply pipeline is a routing decision more than a comprehension one. Command parsing and routing are much simpler against a short piece of text than against raw bytes, and the original attachment is still handed to the model afterwards, so nothing is lost by doing it. The stage is deliberately best effort, and errors never block a reply, which means a flaky transcription service degrades the experience instead of taking the assistant down. The same caution shows up in the prompt budget, where extracted file text is fenced as untrusted external content with explicit boundary markers and a source line, and the long security banner is left out on purpose to keep the media prompt short, a trade off that reads better alongside the rest of the media stack (How the media tools fit together). Every attachment candidate also ends with one recorded disposition, whether it was handled, handed to native vision, dropped past the attachment limit, disabled, missing a model, denied by scope or simply failed, so there is no silent gap. Running a strong multimodal model over every inbound clip is not free either, which is one reason the byte and character caps ship conservative and the attachment policy defaults to a single file (Diali plans). The result is a stage you can widen deliberately rather than one you have to trust blindly.

On Diali

Diali hosts Hosted OpenClaw on Diali so that each customer runs their own assistant, with state on a persistent volume; daily snapshots and one-click restore are available through the Backups add-on (included on Max). If you are working out what enabling media understanding across every channel would mean for you, Diali pricing is the place to look.

• Summaries feed routing; the original media still reaches the model.
• Auto detect tries configured models, then providers, then local binaries.
• Caps ship small: ten, twenty and fifty megabytes for image, audio and video.