local-game-mcp

A universal game-input MCP server for Windows. It lets an AI agent (Claude, or any MCP client) play local games through the same input layer a human uses: scan-code keyboard events, mouse, timed macro sequences, and fast DXGI frame capture — plus per-game profiles that turn raw input into semantic commands like action("end_turn") or action("move_to_tile", {q: 3, r: -1}).

Games see ordinary HID-level input. No screenshot-and-pixel-hunt loop, no game-engine coupling.

See DESIGN.md for the full architecture.

Status

Phase	Scope	State
0	Scaffold, capture, window info	✅
1	Scan-code keyboard, mouse, macros, timing engine	✅
2	Profiles, hex/square/iso grids, semantic actions	✅ (Civ VI profile needs in-game calibration)
2.5	Live OCR perception (Windows.Media.Ocr)	✅
3	Virtual gamepad (ViGEmBus)	planned
4	Game-state bridges (Lua/mod → JSON)	planned

Requirements

• Windows 10/11
• Python 3.11+ (3.12 recommended)
• The game running non-elevated on the primary monitor, preferably in borderless-windowed mode

Install

git clone https://github.com/rrhoopes3/local-game-MCP.git
cd local-game-MCP
py -3.12 -m venv .venv
.venv\Scripts\pip install -e .[dev]
.venv\Scripts\pytest        # should be green

Hook into Claude Code

claude mcp add local-game -- "<absolute path>\.venv\Scripts\local-game-mcp.exe"

Hook into Claude Desktop

{
  "mcpServers": {
    "local-game": {
      "command": "<absolute path>\\.venv\\Scripts\\local-game-mcp.exe"
    }
  }
}

Tools

Input — press_keys (sequences + shift+enter combos), key_hold, type_text, mouse_move, mouse_click, mouse_drag, scroll, macro (fully timed low-level sequences), wait.

Perception — capture (region-cropped, downscaled JPEG), probe_pixels (cheap RGB point checks), check_probe, window_info, plus OCR via the built-in Windows engine (no cloud, ~tens of ms): read_text (text + clickable word boxes), read_hud (all profile [text_regions] as one labeled dict — exact state for ~50 tokens instead of a screenshot), and wait_for_text (blocks server-side until a regex appears in a region — end the turn, wake up when the turn counter changes; no polling through the model).

Profiles — load_profile, list_profiles, action, focus_game.

Raw tools always work without a profile. Coordinates use three spaces: screen (desktop px), window (0–1 over the game's client rect, resolution-independent), and grid (tile coordinates — axial hex, square, or isometric — defined per profile).

Profiles

One TOML file per game in profiles/. profiles/civ6.toml ships as the reference; its hotkeys are the verified game defaults, while grid origin/ size and UI points are marked CALIBRATE (they depend on resolution and UI scale — capture a frame, measure, fill in).

Scope & fair play

This project targets single-player, local, and moddable games. Kernel anti-cheat systems detect synthetic input and virtual drivers; using this in online competitive games violates game ToS and risks bans, and no detection-evasion features will be added. For the planned Mechabellum profile that means vs-AI / practice modes.

Known limitations (v1)

• Primary monitor only for capture.
• Mouse-look in some Raw-Input engines ignores synthetic relative moves — the phase-3 gamepad backend is the fallback for camera control.
• Windows silently drops input to elevated windows (UIPI); window_info reports elevation so the server warns instead of misfiring.