Command line

Every screenpilot command and its useful flags, grouped by job: record, edit, render, combine recordings, drive the desktop and set up, with the full workflow in one block.

The command line does the same steps as the MCP tools. Run it as screenpilot, or as node dist/cli.js from a source checkout. Most people never need it, because their AI app calls the tools for them. Use the CLI to script a render, to check a machine, or to repair a setup.

Run screenpilot --help for the list of commands and screenpilot <command> --help for the flags of one command. Commands print their result as JSON on stdout and progress messages on stderr.

The whole workflow#

This block takes a recording to a finished video. Replace the paths and the voice with your own.

# 1. Record the screen. Press Enter or Ctrl+C to stop.
screenpilot record

# 2. Find idle time and clicks, and get a contact sheet image.
screenpilot analyze sessions/2026-10-27_14-02

# 3. Look closer at moments, with a 0..1 coordinate grid.
screenpilot frames sessions/2026-10-27_14-02/video.mkv --at 1,4.5,10 --grid

# 4. Write a starter storyboard (idle time cut, click zooms filled in).
screenpilot init sessions/2026-10-27_14-02 --suggest --voice ar-EG-SalmaNeural

# 5. Edit storyboard.json, then check it.
screenpilot validate sessions/2026-10-27_14-02/storyboard.json

# 6. Render a fast draft and look at frames of the result.
screenpilot render sessions/2026-10-27_14-02/storyboard.json -q draft
screenpilot check sessions/2026-10-27_14-02/storyboard.mp4 --at 2,8,15

# 7. Render the final video.
screenpilot render sessions/2026-10-27_14-02/storyboard.json -q delivery -o demo.mp4

Record and look#

record#

Records the screen and logs pointer events. Press Enter or Ctrl+C to stop, or pass --duration. It prints the session folder with the video, events.jsonl and windows.jsonl. Recording is native by default, so OBS is not needed.

remap#

screenpilot remap <session> --screen <name> re-maps the pointer events of a session to another monitor. Use it when record warns that it guessed which monitor OBS records, and the guess was wrong. The monitor names are in the session's meta.json under screens.

analyze#

screenpilot analyze <session-dir|video> finds idle spans and clicks, suggests segments and zooms, and writes a contact sheet image of labelled thumbnails. Open the contact sheet first to understand the story of the recording.

frames#

screenpilot frames <video> saves still frames of the raw recording so you can see it. Times are source seconds.

Edit#

init#

screenpilot init <session-dir|video> writes a starter storyboard.json. It saves the file in the session folder, or next to the video, and it picks up events.jsonl when it exists.

validate#

screenpilot validate <storyboard> checks the file against the schema and the source video. It fails with the list of schema issues, or prints valid: true with warnings such as overlapping segments, items past the end of the video and callouts without x and y.

schema#

screenpilot schema prints the storyboard JSON Schema. It is the authority on every field.

voices#

screenpilot voices lists the voices of one provider. With edge (the default) it needs a network connection. With local it lists the installed models and their voices. With command it lists the voices defined in voices.json. Voices explains the providers.

screenpilot voices --provider local
screenpilot voices --provider edge --lang ar

voice-setup#

screenpilot voice-setup [voicetut|namaa] installs a local voice model. The default is voicetut. It needs uv, which also fetches Python 3.11. It creates a Python environment, installs PyTorch and third-party code pinned to tested commits, downloads the model, speaks one test line, and saves the Python path in voices.json. Run it in a terminal: it downloads several GB, and there is no MCP tool for it on purpose. Voices has the sizes, the folders and the steps.

Render and check#

render#

screenpilot render <storyboard> makes the voice, cuts the video and renders an MP4. It takes minutes. It prints the output path, the duration, the freezes it added (autoHolds) and warnings. When a narration line is longer than its slot, the render freezes the video so lines never overlap.

Partial renders#

To review one stretch of a long video, render only that window with --from and --to. Both are output seconds, the same seconds check uses: look at the draft, find the problem at 12 to 20 seconds, fix it, and render just that window again. You can give only --from (to the end) or only --to (from the start).

screenpilot render storyboard.json -q draft --from 12 --to 20

The file is <storyboard>.part-12-20.mp4 (part-12-end when you give no --to), built in <storyboard>.part-12-20.build. It is a short render of just that window, so it is much faster than the whole video. Items outside the window are left out, and the music level, including ducking, carries into it.

Several formats#

--formats renders the same storyboard once per output size, so one edit gives you a YouTube cut, a Reels cut and a feed cut. The voice is generated once for all of them.

screenpilot render storyboard.json --formats 16:9,9:16,1:1 -o demo

• Width and height must be even. Duplicate formats are dropped.
• -o becomes the base name: -o demo writes demo.16x9.mp4 and so on. Without -o, the base is the storyboard name.
• --build-dir cannot be combined with --formats. Each format has its own <name>.<format>.build folder.
• --from and --to work together with --formats: the files become <name>.9x16.part-12-20.mp4.

Quality levels#

build#

screenpilot build <storyboard> does everything except the final render: it generates the voice, cuts the video and writes the HyperFrames project. It prints the build folder, the output duration, automatic freezes and warnings. It accepts --build-dir <dir>.

preview#

screenpilot preview <storyboard> builds the storyboard and opens HyperFrames Studio, where you can scrub through the video.

check#

screenpilot check <video> pulls frames from a rendered video so you can review the result. Times are output seconds. It prints the duration, whether the video has audio, and the frame files.

Combine recordings#

These commands join several storyboards, each with its own recording, into one video. The common use is to insert a new feature clip into the middle of an existing demo without re-recording it. Combine recordings explains the file format, ranges and transitions with a worked example.

sequence#

screenpilot sequence <sequence.json> renders every part and joins them into one MP4. Unchanged parts are reused from the previous run. It prints the output path, the duration, how the parts were joined and each part.

sequence-validate#

screenpilot sequence-validate <sequence.json> checks the sequence and every storyboard it uses. It fails with the schema issues (each names the part, for example parts[1] (feature.json)), or prints valid: true with the warnings of every part.

sequence-schema#

screenpilot sequence-schema prints the JSON Schema of sequence.json.

See and drive the desktop#

These commands let you look at the desktop and test a flow by hand before you write a take. The agent uses the same actions through the MCP tools. They use the real mouse and keyboard, so keep your hands off while one runs. Agent drive explains the whole loop, and Safety explains the limits.

screenshot#

screenpilot screenshot saves a picture of each monitor.

windows#

screenpilot windows lists open windows with their position. On desktops without window control (GNOME and other Wayland desktops) the list is empty.

browser#

screenpilot browser opens, or reuses, the clean demo browser and places it on a monitor. Flags: --screen <name> and --url <url>.

drive#

screenpilot drive has four subcommands. Positions are screen,x,y with x and y from 0 to 1, for example eDP-1,0.5,0.5.

perform#

screenpilot perform <take.json> performs a take on the real desktop while recording, then writes a session folder with the video, events and a ready storyboard. If a step fails, it prints the step number, the step and the error. Press Ctrl+C to stop a run.

take-schema#

screenpilot take-schema prints the JSON Schema of take.json.

Set up and check this machine#

setup#

screenpilot setup copies the app to its folder, adds ScreenPilot to every AI app it finds, and prepares the render browser. Restart the AI app afterwards. Config files are backed up before they change.

doctor#

screenpilot doctor checks this machine: ffmpeg, desktop, monitors, OBS, the demo browser and voices, including a "local voice" check that is skipped when you have not set up a local voice (it is optional). It prints a JSON report and a one-line summary. The exit code is always 0, so read the report. Add --input to also test input injection; this moves the real pointer for a moment.