Platforms

What ScreenPilot can do on each desktop: recording, driving, window control and click logging, plus requirements, per-OS notes and limits.

ScreenPilot picks the right backend for your desktop by itself. Recording works on every supported desktop. Whether the AI can use your apps for you (move the mouse, click, type) depends on the desktop. Ask your AI app to run the ScreenPilot doctor to see which backend it chose and what works on your machine.

Platform support#

The name in brackets is the backend ID that ScreenPilot reports.

Desktop	Record (native)	Drive (pointer, keys, typing)	Windows (list, focus, fit to 16:9)	Pointer and click logging
KDE Plasma, Wayland (`kde`)	PipeWire portal streams + GStreamer	RemoteDesktop portal (approve once)	KWin scripts	KWin pointer; clicks need the `input` group
Any X11 session (`x11`): GNOME on Xorg, Linux Mint Cinnamon, MATE or Xfce, KDE X11	ffmpeg `x11grab`	XTest (any Unicode text, no clipboard or layout switch)	EWMH window manager	XQueryPointer + global hook
GNOME and other Wayland desktops (`portal`)	PipeWire portal streams + GStreamer (asks once)	RemoteDesktop portal (approve once)	Not available: focus and fit steps and `{web}` targets fail with a clear error; window-framed views fall back to monitor views	None, because Wayland hides the pointer; performed takes still log their own clicks
sway, Hyprland and other wlroots desktops (`portal`)	ScreenCast portal + GStreamer (asks once)	Not available (no RemoteDesktop portal)	Not available	None
Windows 10 and 11 (`windows`)	ffmpeg `gdigrab`	`SendInput`	Win32	Cursor polling + global hook
macOS 12 and later, Apple Silicon and Intel (`macos`)	ffmpeg `avfoundation` (Screen Recording permission)	CGEvent (Accessibility permission)	Accessibility API	Quartz polling + global hook (Input Monitoring)

To force a backend, set the SCREENPILOT_DESKTOP environment variable to kde, x11, portal, windows or macos.

KDE Wayland is used daily. Windows, X11 and macOS each have an automated test on a real machine in CI. GNOME and Hyprland have not been run on real hardware yet.

Requirements#

The Windows, macOS and Linux downloads include Node and ffmpeg, so you need nothing else for them. This list is for people who run ScreenPilot from source.

Node 22 or newer and pnpm.
ffmpeg and ffprobe on your PATH.
A screen recorder. The built-in one is the default. On Wayland (KDE, GNOME, wlroots) it records the PipeWire streams from the screen-cast portal with GStreamer, which needs gst-launch-1.0 with pipewiresrc and an H.264 encoder (x264enc, openh264enc or VA-API). On X11 and Windows it records with ffmpeg. On macOS it uses ffmpeg with avfoundation.
Optional: OBS Studio 28 or newer with the WebSocket server enabled, if you prefer OBS's encoders. Use --recorder obs.
Internet access for the default voiceover (Edge voices) and for its voice list. Local voices need none once installed, but installing them needs uv and the internet (see Voices).
wl-clipboard (wl-copy) to type non-Latin text on Wayland. X11 and Windows type any Unicode directly.

If a GStreamer piece is missing, screenpilot doctor prints the install command for your distribution. For example:

# Fedora
sudo dnf install gstreamer1 pipewire-gstreamer gstreamer1-plugin-openh264
# Debian, Ubuntu
sudo apt install gstreamer1.0-tools gstreamer1.0-pipewire gstreamer1.0-plugins-ugly

Linux notes#

Permission prompt#

On KDE and GNOME with Wayland, driving uses the RemoteDesktop portal. Approve the "remote control" dialog once and select all monitors. ScreenPilot saves the approval, so later runs are silent. Recording on Wayland asks once to share the screen (on KDE it reuses the remote-control approval).

KDE Wayland: the input group#

Wayland does not let apps watch global input. To log your mouse clicks, ScreenPilot reads mouse buttons from /dev/input, which needs your user to be in the input group:

sudo usermod -aG input "$USER"   # then log out and back in

Members of the input group can read all keyboard and mouse input on the computer, including passwords typed in any app. Do this only on a computer and account you trust. You can remove yourself afterwards with sudo gpasswd -d "$USER" input.

Without the group you can still record. You lose click ripples, and zoom suggestions fall back to where the pointer came to rest.

X11#

X11 supports everything, including web targets and window framing, and needs no dialog. With fractional or HiDPI scaling, the recording is in X pixels as reported by XRandR.

Windows notes#

Driving uses SendInput. A window you run as administrator cannot be controlled by a normal process. Run ScreenPilot as administrator too.
Coordinates are physical pixels.
Recording is built in (ffmpeg gdigrab, mouse pointer included). OBS is optional.
The demo browser is Chrome, Edge or Brave, whichever is found first. Set SCREENPILOT_BROWSER to pick another.
Holding back or closing notification popups does nothing on Windows.
If your antivirus asks about ScreenPilot, allow it. It watches clicks while recording.

macOS notes#

macOS 12 and later, on Apple Silicon and Intel. Recording, screenshots, driving (pointer, clicks, typing including Arabic and emoji, shortcuts), the window list, focusing and placing windows, and the demo browser all work without OBS. Web demos need Chrome, Edge or Brave in /Applications.

macOS grants privacy permissions to the app that starts ScreenPilot (Claude, Cursor, VS Code, Terminal), not to ScreenPilot itself. Turn that app on in System Settings, Privacy & Security:

Permission	Needed for	Without it
Accessibility	Moving the mouse, clicking, typing, focusing and resizing windows	Driving fails, and the error lists these steps
Screen & System Audio Recording (Screen Recording on macOS 14 and older)	Recording, screenshots, window titles	Recording fails, and the error lists these steps
Input Monitoring	Logging your own clicks while you record	Recordings have no click zooms (optional)

Then quit and reopen that app. See macOS permissions for the steps.

Coordinates are points, not pixels. A Retina display is reported at its "looks like" size, and recordings keep full Retina detail (capped at 4096x2304 for 5K and multi-monitor canvases).
Monitors are named as in System Settings, such as "Built-in Retina Display" or "DELL U2720Q". Identical models get " (2)".
Shortcuts are literal. ctrl+c is Control-C. Use cmd+c for Command-C.
Turn on Do Not Disturb yourself before recording. macOS has no way for ScreenPilot to hold notification banners.
Minimized windows and windows on other Spaces are not listed. Full-screen windows leave full screen when a take places them.

Limitations#

Driving the desktop works on KDE Wayland, X11, GNOME Wayland (without window control), Windows and macOS. It does not work on wlroots desktops (sway, Hyprland). Recording still works there.
Wayland desktops other than KDE expose no window list and no pointer position. That means no window framing, no {web} or {window} targets, and no pointer log for your own recordings.
The default Edge voices need the internet, and they are an unofficial service that may change.
Local voices (VoiceTut and NAMAA) are tested on Linux with an NVIDIA graphics card only. Windows, macOS and CPU-only machines are not tested.
Rendering is not fast. Use a draft render while you iterate.
Narration inside cut parts of the video is dropped, so lines must sit inside the parts you keep.

Back to Install or on to the Safety guide.