Computer use

Letting a model move a real pointer on a real desktop is a large grant, so it is worth knowing exactly what has to be true before a single click happens. OpenClaw makes eligibility capability-based: the connected node has to advertise both a desktop action command and a screen snapshot command, and the node descriptor decides which action families the tool exposes. The agent emits one uniform command and cannot choose how the node fulfils it. Everything else is policy: tool exposure, pairing approval, the node-local setting, and the operating system permissions.

What must be true

• The tool needs a paired, connected node advertising both the desktop action command and screen capture, with the snapshot returning a display frame identifier, plus a vision-capable agent model and a tool policy that exposes the computer tool, which the default coding profile does not.
• On macOS the app setting that allows computer control defaults on but stays off once explicitly turned off, and the fulfiller needs Accessibility and Screen Recording granted to OpenClaw, with the native Peekaboo path also requiring event posting access for its CoreGraphics input primitives.
• Windows and Linux use the optional experimental cua-computer plugin on Windows x64 or ARM64 and on glibc-based Linux x64 or ARM64, whose package includes the pinned CUA Driver SDK runtime, with no separate driver executable, daemon or MCP server configured.
• Sandboxed agents need the computer tool allowed twice, once in the ordinary tool policy and once in the sandbox tool policy, while a cloud session bound desktop is already included in its default sandbox policy and explicit allowlists and denies still apply.

The Gateway is the authorization chokepoint; the driver is a dumb effector.

Frames, coordinates and observations

The computer tool takes one action per call, and coordinates are non-negative integer pixels in the most recent screenshot, which the node maps to display points. Coordinate actions have to echo the frame identifier returned by that screenshot, and an explicit screen index must match the same frame. OpenClaw also carries a node-issued display identity from the screenshot into the action, so a display reconnect or a geometry change fails closed instead of quietly retargeting the same index. These checks reject guessed tokens and tokens from another delivered frame or display, but a token is not a freshness guarantee, because applications can change pixels on the same display after capture. Window input follows the coordinate space reported by the observation: CUA reports image pixels, including when OpenClaw resized the image, while Peekaboo reports desktop logical points and accessibility element bounds keep their native screen coordinates. A desktop frame cannot stand in for a window observation, since the two images can use different coordinate spaces. After window input, providers that expose window state return the action outcome followed by a fresh observation, so the next action can reuse its element references without a separate observation call. Other input actions return a fresh desktop screenshot, and when the screen is pixel-identical to the previous frame the tool returns metadata only, keeping duplicate screenshots out of model context. Screenshots are kept model-only and are never auto-delivered to the chat channel.

Providers and refusals

• Provider selection never falls back per action, so switching providers closes the active execution surface, rotates the provider generation and re-advertises the node commands, and a CUA failure becomes an unavailable result rather than the same action quietly running through Peekaboo.
• The CUA provider adds a browser family bound through opaque browser, page, observation and element references that belong to one execution and driver generation, so navigation invalidates page-element observations and a driver restart invalidates the complete browser reference set.
• The experimental Windows and Linux fulfiller controls only the primary display and rejects held input as well as modifier-held clicks, scrolling and dragging because the typed desktop methods accept no modifiers, while digit and punctuation key presses are rejected and should be sent through the type action instead.

Computer use sits on top of two things that exist before it: the Paired nodes that pair with the Gateway, and the Approvals that let a newly advertised command surface be used at all. A new command forces re-pairing, and a denied command is withheld from the node advertised surface together with its computer capability.

Why the driver stays dumb

OpenClaw deliberately leaves the managed daemon unceilinged and authorises computer use above it, through tool exposure, the dangerous-command allowlist, device and command pairing approval, node-local provider enablement and operating system permissions. The reason is that the pinned driver fixes its permission mode and bounded manifest when the runtime starts, so exact process and window grants would have to be declared in advance, and an ask entry is a hard denial for unattended dispatch. OpenClaw instead drives applications, windows and elements discovered while the agent is running, which bounded mode cannot express without duplicating Gateway policy or preauthorising broad application classes. The socket is created owner-only inside a random owner-only directory, which excludes remote clients and other local users but does not isolate processes running as the same logged-in user, and that boundary is where a Sandboxing does its work. Loopback is reachability rather than identity, so a client does not receive write scope merely because it arrived over a local address; it still has to authenticate and pass device pairing and scope approval. The node-invoke policy also classifies exact arguments before transport dispatch, separating high-risk families such as forced app termination, browser navigation, browser downloads, browser file inputs, recording start, trajectory replay and desktop-scope escalation from ordinary observation and input, which is the part of Security model most worth reading twice.

On Diali

On Diali each customer runs their own assistant, and the Hosted OpenClaw on Diali runtime configuration is generated from the dashboard and replaced at each release. State lives on a persistent volume, with daily snapshots and one-click restore available through the Backups add-on (included on Max). Plan details are on Diali pricing.

• Eligibility is capability-based: no advertised commands, no computer tool.
• Coordinates are bound to one frame, so a stale reference fails closed.
• Authorisation lives in the Gateway, not in the driver.