jetkvm-mcp

Give an AI eyes and hands on a physical computer.

jetkvm-mcp is an MCP server that turns a JetKVM — a small open-source KVM-over-IP device — into a machine that Claude (or any MCP client) can see and operate directly: watch the screen, type, click, mount boot media, and control power. Because the JetKVM sits on the HDMI and USB ports, the AI drives the computer below the OS — BIOS screens, bootloaders, installers, headless boxes with no network, machines that are wedged. No agent, no SSH, nothing installed on the target.

you:    "Screenshot the machine. It's stuck — what's wrong?"
claude: → screenshot → "It's sitting at a GRUB rescue prompt. The root partition
         UUID changed. Want me to boot it manually?" → type_text → enter → fixed

Works against stock JetKVM firmware — no modifications to the device.

The two planes

Plane	Tools	Nature
Screen control (eyes + hands)	screenshot, click, double_click, move_mouse, type_text, press_key, scroll	vision loop — the AI looks, then acts
Device control	mount_media_url, mount_media_storage, upload_media, upload_and_mount, unmount_media, list_storage, delete_storage_file, storage_space, virtual_media_state, power, power_state, dc_power, wake_host, wol, usb_emulation, video_state, reboot_device	deterministic RPC

Full parameter reference: docs/tools.md.

How it works

One WebRTC peer connection to the device drives everything:

Claude ──MCP/stdio──▶ server.py (this repo, runs on your workstation)
                        │
                        └──WebRTC over LAN──▶ JetKVM ──HDMI-in / USB-HID-out──▶ target machine
                             ├─ H.264 video track ─▶ decoded locally (PyAV) ─▶ JPEG screenshots
                             └─ "rpc" data channel ─▶ keyboard / mouse / media / power JSON-RPC

• The device already streams its HDMI capture as an H.264 video track — the client decodes it locally and hands the AI JPEG snapshots on demand. (JetKVM has no snapshot endpoint; it doesn't need one.)
• A reliable rpc data channel carries every JSON-RPC method the device's own web UI uses: keyboardReport, absMouseReport (absolute 0–32767, drift-free), mountWithHTTP, setATXPowerAction, and friends.
• The server connects lazily on the first tool call and keeps the one connection alive.

Deep dive — handshake, codec negotiation, the keyframe/PLI story, coordinate mapping: docs/architecture.md.

Requirements

• A JetKVM attached to the target machine, reachable on your network
• Python 3.11+ on the machine that runs Claude
• aiortc/av wheels bundle FFmpeg on macOS/Linux; if a build from source is triggered, install FFmpeg dev libraries first (brew install ffmpeg / apt install libavdevice-dev)

Quickstart

git clone https://github.com/shvartzj1/jetkvm-mcp.git
cd jetkvm-mcp
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env        # set JETKVM_URL (+ JETKVM_PASSWORD if your device has one)

Prove the pipeline before wiring it into anything — this connects, holds the stream open, and saves four screenshots:

set -a; source .env; set +a
python smoke_test.py

Expected output — sustained ~60 fps, snapshots in single-digit milliseconds after the first:

connected. video_state: {'ready': True, 'width': 1280, 'height': 1024, 'fps': 60}
  snapshot 0: 1280x1024   179578 bytes  (grab 3129 ms)  frames_seen=1
  snapshot 1: 1280x1024   178280 bytes  (grab   11 ms)  frames_seen=128
  ...

Wire it into Claude

Claude Code (one command, available in every session):

claude mcp add jetkvm --scope user \
  --env JETKVM_URL=http://192.168.1.50 \
  --env JETKVM_VERIFY_TLS=false \
  -- /abs/path/jetkvm-mcp/.venv/bin/python /abs/path/jetkvm-mcp/server.py

Claude Desktop (claude_desktop_config.json):

{
  "mcpServers": {
    "jetkvm": {
      "command": "/abs/path/jetkvm-mcp/.venv/bin/python",
      "args": ["/abs/path/jetkvm-mcp/server.py"],
      "env": {
        "JETKVM_URL": "http://192.168.1.50",
        "JETKVM_PASSWORD": "",
        "JETKVM_VERIFY_TLS": "false"
      }
    }
  }
}

Then just talk to it: "Screenshot the machine, open a terminal, and check disk usage." The AI calls screenshot → reasons → click / type_text → repeats.

The killer workflow: hands-free bare-metal provisioning

Device control and screen control compose into something no in-OS agent can do — installing an operating system on an empty machine:

mount_media_url("https://mirror.lan/rocky-9.iso", "CDROM")   # host the ISO yourself
power("reset")                                               # reboot into the installer
# screenshot → click → type_text … the AI walks through the installer by sight
unmount_media()

The device's own storage partition is tiny, so mount_media_url (the device streams the image over HTTP with range requests) is the right path for full-size ISOs; upload_and_mount is for small recovery images.

Gotchas (read this before filing a bug)

• First screenshot takes ~3 s; the rest are instant. The device only emits an H.264 keyframe when asked via RTCP PLI. Browsers request keyframes automatically; aiortc does not — so this client sends PLI on connect and whenever frames go stale (_request_keyframe in client.py). Without that, decode fails on every packet forever (avcodec_send_packet: Invalid data). If you're building your own client: this is the trap.
• Keyboard layout: type_text maps ASCII → USB HID usage codes assuming the US layout on the target OS. On other layouts, shifted symbols swap (on a UK target, " arrives as @). Letters, digits, and / - . ; = are layout-stable; prefer them in critical commands.
• Coordinates: click/move_mouse take pixel coordinates on the most recent screenshot; the client maps them to the HID absolute range using the live frame dimensions, so there is no drift.
• getVideoState may report streaming: 0 even while frames flow at 60 fps — cosmetic quirk, ignore it.
• TLS: stock firmware serves plain HTTP on the LAN. The device supports optional TLS (Settings → Advanced) — enable it and set JETKVM_URL=https://…, plus JETKVM_VERIFY_TLS=true if the cert is trusted. WebRTC media/control is DTLS/SRTP-encrypted peer-to-peer regardless of how the signaling travelled.
• power needs the ATX extension board wired to the motherboard header; without it the tool is a no-op (power_state reads power: false).

Safety

This lets a language model drive a real computer with real consequences. Recommendations:

• Point it at a test box or lab machine first, not your production NAS.
• The destructive tools are power, reboot_device, dc_power, mount_*, delete_storage_file, and any press_key of a reboot chord — consider requiring per-call confirmation for them in your MCP client's permission settings.
• Set a device password (and TLS) if the JetKVM is reachable by anyone but you.

Development

jetkvm/client.py   WebRTC + JSON-RPC client (connect, snapshot, HID input, uploads)
jetkvm/keymap.py   ASCII / key-combo → USB HID usage codes
server.py          FastMCP server exposing the 24 tools
smoke_test.py      live end-to-end check against a real device
docs/              architecture + tool reference

Validated end-to-end against a JetKVM v2 on firmware/app 0.5.8 (Jul 2026): sustained 60 fps decode, keyboard input, HTTP CDROM mount/unmount, ATX/DC state reads — including driving it from a live Claude session. The RPC surface is verified against the jetkvm/kvm source.

Ideas / roadmap

• Keyboard layout profiles for type_text (US hardcoded today)
• Gate destructive tools behind an env flag
• Native getSnapshot RPC upstream in the firmware would remove the H.264 decode dependency entirely (see jetkvm/kvm#1459)

License

MIT. Not affiliated with JetKVM/Improve Robotics — this is an independent client of the device's public API.