Skip to content
Guides

Talk mode

Continuous speech conversations across native recognition, client-owned realtime sessions and a Gateway-owned relay, with the catalogue rather than local guesswork deciding what a given client can actually run

7 min read

Voice looks simple from the outside: you talk, the assistant answers. Underneath, Talk mode is several runtime shapes that differ in who owns the audio, who owns the tools and the transcript, and who holds the provider credential. Native Talk is a continuous loop that listens for speech, sends the transcript to the model through the active session, waits for the response, then speaks it through the configured provider. Realtime paths replace parts of that loop with a live provider connection, and which of them a client may use is decided by the Gateway catalogue rather than by the client guessing.

The runtime shapes

  • Native Talk on macOS, iOS and Android uses platform speech recognition, Gateway chat and a speak command, with Apple Speech possibly using network services on macOS and iOS and Android behaviour depending on the installed speech service, while nodes advertise a talk capability and declare which talk commands they support.
  • On iOS, client-owned WebRTC is used for OpenAI realtime configurations that select the WebRTC transport or omit transport, while explicit relay, provider-websocket and non-OpenAI realtime configurations stay on the Gateway-owned relay and non-realtime configurations use the native speech loop.
  • Standalone Talk on Apple Watch uses native WebRTC and Opus over UDP with Gateway-owned call control, takes the Gateway configured realtime provider, keeps tools and transcript ownership on the Gateway, and fails visibly when a configuration is unsupported.
  • Android uses Gateway-owned relay realtime only when the catalogue reports the realtime group ready and the configured model also passes the Android client gate, never opens a client-owned WebRTC session, and deliberately keeps the newer live models on native speech recognition until the relay path is proven live from an Android device.
Unsupported configurations fail visibly without a relay fallback.

Credentials, routes and voices

The Gateway-owned WebRTC route keeps OAuth and Platform credentials away from relay clients, and backend WebSocket paths keep the Platform key on the Gateway. For the generally available realtime models, browser sessions prefer Platform credentials in a fixed order: the configured realtime API key, then an OpenAI API-key profile, then the environment variable. With none of those configured, browser Talk falls back to an OpenClaw ChatGPT OAuth profile and exchanges session descriptions through the Gateway single-use offer broker, so the OAuth token never reaches the browser. A configured Platform credential that cannot be resolved fails closed instead of silently falling through to OAuth. The two live routes differ as well: the public API route requires a Platform key, while the Codex route prefers an OpenClaw ChatGPT OAuth profile and falls back to Platform API-key authentication. Their voice families differ too, so choosing the same model and a supported voice in Talk and in Discord is what makes them sound the same, and a Codex-route voice selected with the public live model falls back to that route default. Account-issued, unlisted routes can be set in configuration but are not published through catalogs or diagnostics, never use OAuth, and require a Platform key. Runtime bounds apply throughout: eight concurrent sessions per Gateway, a thirty-minute session lifetime, and sixty-second single-use offer tokens for browser sessions. A rejected session does not identify the cause by itself, so check the selected account, model and voice.

Voice directives and limits

  • A reply can carry a single JSON line as its first non-empty line to control voice, and that line is stripped before playback, unknown keys are ignored, and a once flag applies to the current reply only while without it the chosen voice becomes the new Talk mode default.
  • Supported directive keys cover voice and model identifiers, speed, rate in words per minute, stability, similarity, style, speaker boost, seed, normalisation, language, output format, latency tier and the once flag, with ElevenLabs accepting stability, similarity and style from zero to one, speed from 0.5 to 2, and latency tier from zero to four.
  • For replies dominated by fenced code, the speak command uses a short spoken message directing the listener to the screen, while inline code and ordinary prose remain part of the spoken reply.

Talk is a node capability before it is a feature, so the Nodes page is where that capability and command model is defined. Discord uses the same Gateway bridges as Talk when it is configured with the corresponding live model, which is where Talk meets Channels. Discord also preserves each speaker identity through delegated OpenClaw work, keeping voice-model connections separate per speaker while shared room context belongs to the agent conversation.

Why the catalogue decides

First-party Talk clients are told to read the catalogue instead of maintaining provider aliases locally, and the reason is that provider identifiers, modes, transports, brain strategies, realtime audio formats and capability flags all change faster than a shipped client does. The catalogue also exposes the runtime-selected readiness result, and an older Gateway that omits group readiness should be treated as unverified rather than definitively unconfigured, which is a deliberate refusal to guess. The same instinct shows in the failure behaviour, where unsupported configurations fail visibly instead of quietly relaying and a configured credential that cannot be resolved fails closed. Routing follows ownership rather than convenience, which is why the agent-consult brain strategy routes realtime tool calls through Gateway policy, while the direct-tools setting exists only for legacy compatibility and the none setting is for transcription or external orchestration. That routing is what keeps a spoken turn inside the same agent run as a typed one, which is also where its Memory lives. It is the reason a voice conversation inherits the rest of the agent boundaries rather than bypassing them, including whatever Sandboxing the agent already runs inside.

On Diali

On Diali, channels are connected from the dashboard and the Hosted OpenClaw on Diali runtime configuration is generated from it and replaced at each release. Each customer runs their own assistant, with state on a persistent volume; daily snapshots and one-click restore are available through the Backups add-on (included on Max). There is more on Diali security.

  • Talk is several runtime shapes, not one, and they differ in who owns what.
  • Read the catalogue; do not keep provider aliases inside the client.
  • Unsupported configurations fail visibly rather than relaying silently.
Get started

Stop reading about it, build one

Set up an agent, pick a channel, and have it working inside the app you already keep open.