Control Runs

One remote.control call is one run: an ordered list of actions against a single node, approved once and executed either to completion or to the first failure. It is the only way an agent reaches a desktop.

This is the mechanics page. For what the capability is and when to reach for it, see Remote Control. For the catalogue of actions a run can contain, see Commands & Actions.

Anatomy of a run

remote.control({
  actions: [
    { action: "app.focus",      params: { name: "chrome" } },
    { action: "mouse.click",    params: { x: 640, y: 480, button: "left" } },
    { action: "keyboard.type",  params: { text: "Hello world" } }
  ],
  observe: "screenshot"
})

Action names are not the same as the command types the node receives. mouse.click travels as input.mouse.click. See Actions and command types.

A one-off is also accepted, as a single action with its params instead of the actions list. It is the same dispatch through the same gates, but none of the batch machinery applies: observe is a batch parameter, so a single action returns its own result and no closing capture. Everything below describes the batch, which is the form to reach for.

Sequential execution

Actions run in order, one at a time. There is no parallelism inside a run, because the actions share one desktop: a click means nothing if the window it targeted has not been focused yet.

The first failure ends the run. Every action after it is reported back as Not executed rather than being attempted, so a half-finished form is never mistaken for a completed one. The agent sees exactly where the run stopped.

The node is pinned once, at the start. A run resolves its target before the first action and stays on that machine, so a run can never silently retarget to another desktop partway through, which would otherwise be possible when the agent has more than one node and the first goes offline mid-run.

Observation

By default a run ends with a fresh screenshot, taken after the last action. This is what lets a model check its own work: it reasons about the state it produced, not the state it started from.

Pass observe: "none" to skip it. Worth doing when the run's effect is not visual, when a screenshot would be noise, or when the screen at that moment holds something you would rather not send anywhere.

Captures carry the geometry of the region they came from, so a coordinate read off a downscaled image maps back to a real pixel. That geometry is emitted on Windows only; see Platform Support & Limits.

A capture is put in front of the model itself, rather than only into the chat transcript, on Claude models reached through the Anthropic API or a connected Claude subscription. On every other provider path — OpenAI-compatible, Gemini, Bedrock, Copilot — the model receives the text result alone and cannot see the image, which is worth knowing before you ask a model to check its own work by looking.

One approval for the whole run

A run's risk is the maximum across its actions. A batch containing one High-risk action is a High-risk run, and approving it once covers everything in it.

The approval is raised once, before anything executes, and it goes to the channel the conversation is actually happening in rather than only to a Dashboard nobody is looking at.

That standing consent is bounded deliberately: it lasts thirty minutes, belongs to one conversation, is held in memory so a Gateway restart clears it, and is dropped when the session's work is cancelled. It also applies only to interactive chat turns — a workflow, a scheduled or cron run, a heartbeat turn, a webhook turn and an MCP call each ask again on every single call. See plan-level consent.

Timeouts

timeoutSeconds bounds each action, not the run as a whole, and is clamped to 1–120 seconds regardless of what is asked for.

A timeout fails that action, which ends the run under the halt rule above. It does not undo what already completed: text already typed stays typed. On Windows a timed-out or cancelled shell command has its whole process tree killed, so a hung command does not leave children running.

Watching and stopping a run

While a run is in flight the Dashboard chat shows it live: the action list, the step in progress, the risk on that step, and the capture it just took.

Stop cancels both halves: the in-flight command on the node, and the agent turn that ordered it. Both are needed — cancelling only the command would let the agent issue the next one immediately, and cancelling only the turn would leave the current command running on the machine. It is not a pause, and not a suggestion the model may decline. Remaining actions report as Not executed, exactly as they would after a failure.

The wire contract is in the node protocol.

When the node is not there

A node is considered offline after two minutes without a heartbeat. A run can be told to fail immediately in that case rather than waiting out the window, which is the default for agent-initiated runs, so you get a clear "that machine is not connected" instead of a timeout thirty seconds later.

What a run can do

A run draws on twenty-eight actions across eight families: screen, input, window, application, clipboard, notification, file, and system. A handful of command types the node implements are deliberately not reachable from a run at all; both lists are in Commands & Actions.

Where to go next

• Commands & Actions — every action, its command type, scope, and risk
• Gates, Limits & Audit — the four checks each action passes
• Approval Gates — how risk tiers decide what gets asked
• Remote Control — the feature, rather than the mechanics