excalidraw-ai — MCP server (AI interface to Excalidraw)

The clean, non-clunky interface for an AI to read, watch, and edit a live Excalidraw room. Instead of browser automation (fiber-read + drag-drop + remote browser drops), it joins the room as a socket-bot over the collab relay and speaks the AES-GCM-encrypted scene protocol directly — deterministic, headless, scalable.

POC status: core bridge + MCP stdio surface working end-to-end against a self-hosted excalidraw-room relay (selftest + MCP smoke pass).

Why a relay we control

The bot talks to a relay. Excalidraw's hosted relay (oss-collab.excalidraw.com) origin-guards non-browser bots (403/400), so for our own rooms we run excalidraw/excalidraw-room (MIT, same code as prod) and point an Excalidraw UI at it via VITE_APP_WS_SERVER_URL.

Architecture

Excalidraw UI (fork) ──socket.io──▶ relay (excalidraw-room) ◀──socket-bot── excalidraw-ai (MCP)
   (humans)                           self-hosted                 (the AI's interface)

Tools

Tool	What it does
create_room	new room id/key for our relay
join_room	join a room, read scene, start listening
leave_room	disconnect
get_scene	board elements + texts
get_changes	add/update/remove deltas since last call ("hear the room")
add_elements	append elements (broadcast live)
update_elements	move / retitle / recolor existing (version bump)
delete_elements	mark deleted

Run

npm install
# relay (one terminal):
cd /tmp/excalidraw-room && PORT=3002 node dist/index.js
# self-test the bridge core:
npm run selftest
# MCP smoke (stdio handshake + join + get_scene):
npm run mcp-smoke
# start the MCP server (stdio) — wire into Hermes/Copilot/Claude:
node src/server.js

Wire into Hermes (~/.hermes/config.yaml)

The server can run two ways:

Local (stdio) — spawn as a subprocess:

mcp_servers:
  excalidraw_ai:
    command: node
    args: ["/root/workspace/mcclawd-org/excalidraw-ai/src/server.js"]

Deployed on a VPS (HTTP / StreamableHTTP) — reach it over the network with token auth. See the Deploy section below.

mcp_servers:
  excalidraw_ai:
    url: "http://<vps>:4000/mcp"
    headers:
      Authorization: "Bearer <EXCALIDRAW_AI_TOKEN>"
    timeout: 90

Restart Hermes → tools appear as mcp_excalidraw_ai_*.

Deploy (VPS, PM2)

Run the server as a headless service next to the relay. Currently deployed to 72.62.115.153 (srv1544821): relay :3002, MCP :4000 (token-authed), under PM2 (excalidraw-room, excalidraw-ai), ecosystem.config.cjs in this repo.

# on the VPS
#   /opt/excalidraw-room  (relay: PORT=3002 node dist/index.js)
#   /opt/excalidraw-ai    (MCP:  node src/server-http.js)
cd /opt/excalidraw-ai && npm ci
# put EXCALIDRAW_AI_TOKEN=<long-random> into /opt/excalidraw-ai/.env (chmod 600)
pm2 start ecosystem.config.cjs && pm2 save && pm2 startup systemd
# firewall: ufw allow 3002/tcp 4000/tcp

Endpoints: GET /health, POST/GET /mcp (StreamableHTTP). Unauthorized requests → 401. Hardening note: TLS via nginx+Let's Encrypt still TODO.

Layout

• src/crypto.js — AES-GCM E2E helpers (Node Web Crypto)
• src/elements.js — element builders + scene merge/diff (id+version semantics)
• src/room.js — RoomClient: socket join, read, write, change feed
• src/app.js — shared tool registration (used by both entrypoints)
• src/server.js — MCP stdio entrypoint
• src/server-http.js — MCP HTTP entrypoint (token auth + /health)
• ecosystem.config.cjs — PM2 config for relay + server (ESM-safe)
• scripts/selftest.mjs, scripts/mcp-smoke.mjs — verification