Safety and privacy

What the agent can touch when it drives your desktop, how to stop it, how to limit it, what is logged, and what leaves your computer.

When an AI app drives your desktop through ScreenPilot, it can do what you can do with a mouse and keyboard: click, type, press shortcuts and open apps. This page explains how to stop it, how to limit it, what gets logged, and what leaves your computer.

Recording your own screen and editing a video do not involve any of this. These controls matter when the agent drives the desktop.

The agent uses your real mouse and keyboard#

There is no sandbox and no virtual screen. Clicks and keystrokes go to whatever window is in front, as if you had made them. If the agent clicks the wrong button in your mail app, that mail app receives the click.

• Before the first driven action, Wayland desktops (KDE, GNOME) show a remote control approval dialog. Approve it once. The approval token is saved in state.json in the config folder (see the audit log for where that is), so later runs are silent. Delete that file and ScreenPilot asks again.
• On macOS the AI app needs Accessibility permission. On X11 and Windows there is no dialog.
• While a person is at the machine, ask them to keep their hands off the mouse during a take.

Move the mouse to stop it#

ScreenPilot watches the pointer while it drives. If the pointer is more than about 40 pixels away from where ScreenPilot just put it, it assumes a person took over and stops.

• Live driving (tools such as click, type_text): the action fails with a message that the pointer was moved by someone else. Nothing else runs until the agent calls resume to carry on from where the pointer is now, or abort to stop. The agent should take a new screenshot after resume, because the screen may have changed.
• A take (perform): the take stops at that step, keeps what was recorded so far, and reports the failed step. Fix the cause and perform it again. A take cannot be resumed.

The check is made before each action and before each typed character. It reacts to the mouse only: pressing keys on your own keyboard does not stop anything.

Abort#

Abort always works, on every desktop.

• From an AI app: the abort tool stops a running perform and closes the live driver. The agent has to call it, so if you need the agent to stop right now, also stop it in the AI app itself.
• From the command line: press Ctrl+C while screenpilot perform runs.
• Quitting the AI app closes the ScreenPilot server, which aborts a running take and releases the mouse.

After an abort the next drive tool opens a fresh driver. Abort does not undo anything the agent already did.

Unattended runs#

If nobody sits at the machine, the pointer check can get in the way: any other program that moves the cursor would stop the agent. For those runs, turn it off:

SCREENPILOT_INTERRUPT=off

With this set, ScreenPilot ignores pointer movement by anyone else. Only abort (or a timeout, see below) stops the agent.

Setting the environment variables#

All limits below are environment variables of the ScreenPilot server, so they must be set where your AI app starts it: in the env of the screenpilot entry in the app's MCP settings.

"env": {
  "SCREENPILOT_INTERRUPT": "off",
  "SCREENPILOT_ALLOW_APPS": "firefox,notepad",
  "SCREENPILOT_PERFORM_TIMEOUT": "180"
}

For Claude Code you can add them when you register the server:

claude mcp add --scope user screenpilot \
  -e SCREENPILOT_ALLOW_APPS=firefox,notepad \
  -e SCREENPILOT_PERFORM_TIMEOUT=180 \
  -- node /abs/path/to/dist/mcp.js

Restart the AI app after a change. Running the ScreenPilot installer again rewrites its entry, so check your variables afterwards. screenpilot setup --print shows the entry the installer writes; see Install.

None of these limits is on by default.

Limit which apps get input#

SCREENPILOT_ALLOW_APPS is a comma-separated list of app names. Before each click, drag, scroll, key press or typing action, ScreenPilot checks which window has focus. If its class or title does not contain one of the names (case does not matter), the action fails and tells the agent to focus an allowed app first.

SCREENPILOT_ALLOW_APPS=firefox,notepad

Limits of this check:

• It needs window control. On desktops that cannot tell which window is focused (GNOME on Wayland), every guarded action fails with an error that says so. Do not set it there.
• It applies to the MCP drive tools. focus_window and pointer_move are exempt, so the agent can move the pointer and focus a window before the check runs.
• It does not apply to perform takes or to screenpilot perform and screenpilot drive on the command line. A take is a file you can read before it runs.
• It checks the focused window, not what is under the pointer. It is a safety net, not a security boundary.

Limit which secrets can be typed#

The agent can type the value of an environment variable without ever seeing it: type_text with { "env": "NAME" }. SCREENPILOT_ALLOW_ENV is a comma-separated list of the names it may use. Any other name is refused.

SCREENPILOT_ALLOW_ENV=DEMO_PASSWORD,DEMO_EMAIL

Without it, the agent can type any variable in the server's environment, and that includes everything your AI app passed to it. The list is checked by the type_text tool only, not by { "type": { "env": ... } } steps inside a take.

Time limit for takes#

SCREENPILOT_PERFORM_TIMEOUT is a number of seconds. When a perform call runs longer than that, ScreenPilot aborts it, so a take that hangs does not hold your mouse forever.

SCREENPILOT_PERFORM_TIMEOUT=180

Set it a little above the length of your take. It limits perform through the MCP server. It does not limit live drive tools or the command-line perform.

The audit log#

Every drive action the agent takes through the MCP tools is appended to a file, one JSON line per action:

{"at":"2026-10-28T09:14:03.221Z","tool":"type_text","args":{"text":{"chars":18}}}

The file is audit.jsonl in ScreenPilot's config folder:

What is logged: the time, the tool name and its arguments for focus_window, pointer_move, click, drag, scroll, key and type_text. Key combos are logged. Actions that the app limit refuses are logged too, because the log is written first.

What is not logged:

• Typed text. Only its length in characters. A secret given as { "env": "NAME" } logs the variable name, never the value.
• The steps inside a take. A perform call is not broken down into its clicks. The take file and the recording are the record of what happened.
• Screenshots, recordings and results. Only the actions.
• Command-line use. screenpilot drive and screenpilot perform do not write to the log.

The log stays on your computer and ScreenPilot never trims it. Delete it when you no longer need it. A failure to write it never stops a take.

Secrets#

• Never put a password or token in take.json as plain text. The file stays on disk and is easy to share by accident.
• Use { "type": { "env": "NAME" } } in a take, or { "env": "NAME" } in type_text. The value is read from the environment when the action runs. It is not logged and not returned to the agent.
• The recording still shows what is on screen. A password in a field that shows plain text appears in the video and in screenshots, which the AI app sees. Use fields that mask input.

setup --allow-tools (Claude Code)#

By default Claude Code asks you before each ScreenPilot tool call. That prompt is a brake on every click. If you prefer to skip it, the installer can allow all ScreenPilot tools at once:

screenpilot setup --allow-tools

What it changes: it adds mcp__screenpilot to permissions.allow in ~/.claude/settings.json. That one rule covers every ScreenPilot tool, including click, type_text, key and perform. Your other entries and comments stay as they are, and the file is copied to settings.json.screenpilot-backup first.

To undo it, either delete the mcp__screenpilot line from that file, or run:

screenpilot setup --remove --clients claude-code

This removes the rule, and also removes ScreenPilot and its skill from Claude Code. Run screenpilot setup again afterwards to add it back without the rule.

Privacy: what stays on your computer#

ScreenPilot needs no account and no API key. Recording, analysis, editing and rendering run on your computer. Files are saved in your ScreenPilot folder inside Videos (Movies on a Mac), or in sessions for command-line use.

What goes over the network:

• The voice. The default voices (edge) are an online service: the text of your narration lines is sent to it, and the audio comes back. Listing Edge voices uses it too. Local voices (local) run on your computer and send nothing. A command voice runs a program you set up yourself in voices.json; a storyboard cannot add one, so a storyboard from someone else cannot run programs. See Voices.
• Your AI app. The agent sees what its tools return: screenshots of your monitors, frames of your recordings, window titles and the text of the demo browser's page. Your AI app sends that to its model provider under its own privacy terms. Do not let the agent look at screens you would not show that provider.
• One-time downloads. The first render downloads the video renderer (about 100 MB). Pages in the demo browser load whatever sites you point it at, and a storyboard that asks for Google Fonts loads them at render time.

ScreenPilot turns off telemetry in the HyperFrames renderer it uses.

Before you let an agent run alone#

1. Rehearse with a dry run and watch it once.
2. Close apps and tabs you do not want touched.
3. Set SCREENPILOT_ALLOW_APPS, SCREENPILOT_ALLOW_ENV and SCREENPILOT_PERFORM_TIMEOUT.
4. Keep secrets in environment variables.
5. Know how to stop it: abort from the AI app, or Ctrl+C for the command line.
6. Check audit.jsonl afterwards.