computer-use-mcp

MCP server for computer use automation. Provides screenshot, click, keyboard, mouse, and drag-and-drop tools. Supports Linux (X11) and Windows (10/11).

System Dependencies

Linux

sudo apt install xdotool scrot x11-xserver-utils

• xdotool — mouse/keyboard automation
• scrot — screenshots
• xrandr (from x11-xserver-utils) — screen resolution

Windows

No external dependencies required. Uses built-in PowerShell with .NET Framework:

• PowerShell — comes pre-installed on Windows 10/11
• user32.dll — native mouse/keyboard input via SendInput P/Invoke
• System.Drawing — screenshot capture
• System.Windows.Forms — screen resolution detection

Requirements:

• Windows 10 or 11
• .NET Framework (pre-installed)
• Desktop session must be active (screen unlocked)

Setup

pnpm install
pnpm build

Usage

With Claude Code

Add to ~/.claude/settings.json (Linux) or %USERPROFILE%\.claude\settings.json (Windows):

{
  "mcpServers": {
    "computer-use": {
      "command": "node",
      "args": ["/path/to/computer-use-mcp/dist/index.js"]
    }
  }
}

Development

pnpm dev

Tools

Tool	Description
screenshot	Capture full screen or a region. Returns optimized JPEG base64.
click	Click at (x, y) with left/right/middle button.
double_click	Double-click at (x, y).
type_text	Type text at current cursor position.
key_press	Press key combinations (e.g. ctrl+c, alt+tab).
mouse_move	Move cursor without clicking.
scroll	Scroll up/down at a position.
drag	Drag and drop from one position to another.
get_screen_size	Get screen resolution.
get_cursor_position	Get current cursor position.
wait	Wait N milliseconds between actions.

Platform Details

Linux

• Uses xdotool for mouse/keyboard, scrot for screenshots, xrandr for screen info
• Requires X11 display (Wayland not supported)

Windows

• Uses PowerShell with inline C# (Add-Type) calling user32.dll SendInput
• ~200-500ms overhead per operation due to PowerShell startup and Add-Type compilation
• Captures primary monitor only
• typeText uses KEYEVENTF_UNICODE for full Unicode support
• delay_ms parameter on type_text is ignored (SendInput sends all chars at once)
• DPI-aware: calls SetProcessDPIAware before coordinate/screenshot operations

Notes

• Screenshots are automatically resized if wider than 1920px and compressed to JPEG quality 80
• A 100ms delay is applied after each action to avoid race conditions
• All actions are logged to stderr in JSON format
• Platform is auto-detected at startup via process.platform