Page Vision MCP

Structured visual analysis pipeline for coding agents — give your AI agent the ability to see.

中文文档

What It Does

Page Vision MCP turns visual analysis from a prompt into a tool. Instead of asking an LLM to "look at this image," your coding agent calls a structured API that returns consistent, machine-readable results with coordinates.

The Problem: AI coding agents modify frontend code but can't verify the result. They guess effects from source code alone.

The Solution: Screenshot → VLM Analysis → Structured JSON with coordinates → Agent makes informed decisions.

5 Tools:

Tool	What It Does	VLM	Playwright
capture_page	Screenshot a webpage (fallback)	-	Yes
analyze_ui	Structured UI analysis with region-level localization	Yes	-
compare_ui	Screenshot diff with mode constraints	Yes	-
extract_text	Block-level OCR with coordinates	Yes	-
crop_analyze	Progressive crop refinement with depth guards	Yes	-

3 Resources: vision://health | vision://version | vision://models

Pipeline

Screenshot ──> analyze_ui ──> Structured JSON
                  │
                  ├── Need detail? ──> crop_analyze (depth +1)
                  ├── Need text?   ──> extract_text
                  └── Need diff?   ──> compare_ui

Typical workflow:

"Check this page" ──> capture_page ──> analyze_ui ──> Agent finds issue
                                                          │
                                                    Agent fixes code
                                                          │
                               capture_page ──> compare_ui ──> Verified ✓

Quick Start

Install

pip install page-vision-mcp

# Optional: Playwright for screenshot fallback
pip install "page-vision-mcp[playwright]"
playwright install chromium

Configure

Set environment variables in your MCP client config (recommended) or create a .env file (see .env.example):

VISION_API_KEY=your-api-key
VISION_BASE_URL=https://api.openai.com/v1
VISION_MODEL_ID=gpt-4o

Works with any OpenAI-compatible VLM: GPT-4o, GLM-4V, Qwen-VL, etc.

MCP Client Config

Add to your MCP client configuration (Claude Code, Trae, Cursor, etc.):

{
  "mcpServers": {
    "page-vision-mcp": {
      "command": "page-vision-mcp",
      "env": {
        "VISION_API_KEY": "your-api-key",
        "VISION_BASE_URL": "https://api.openai.com/v1",
        "VISION_MODEL_ID": "gpt-4o"
      }
    }
  }
}

See examples/mcp_config.json for the full configuration with all options.

Tool Details

analyze_ui — UI Analysis

analyze_ui(
    image_source="screenshot.png",  # URL/path/base64/data_url
    prompt="",                       # optional focus
    task="ui",                       # ui/screenshot/qa/general
    detail="high"                    # low/auto/high
)

Returns structured observations with region-level localization (page → section → component → text), inferences, uncertainties, and suggested regions for further analysis.

Example output:

{
  "status": "success",
  "result": {
    "summary": "A dashboard with navigation bar, sidebar, and content area",
    "observations": [
      {"type": "component", "text": "Top navigation bar with 3 menu items", "bbox": [0.0, 0.0, 1.0, 0.08]},
      {"type": "issue", "text": "Menu items too close together", "bbox": [0.15, 0.02, 0.6, 0.07]}
    ],
    "suggested_regions": [
      {"description": "Menu overlap area", "bbox": [0.15, 0.02, 0.6, 0.07], "reason": "Text may overlap, zoom in"}
    ]
  }
}

compare_ui — Screenshot Diff

compare_ui(
    image_before="before.png",
    image_after="after.png",
    mode="viewport",     # viewport/fullpage/component
    focus="changes"      # changes/layout/text/colors/general
)

Returns structured differences (added/removed/modified/moved) with coordinates. Perfect for verifying code changes.

extract_text — OCR

extract_text(
    image_source="screenshot.png",
    language="auto",        # auto/chinese/english/japanese/korean
    preserve_layout=True
)

Returns text blocks with type, confidence, and bounding box coordinates. Preserves original language.

crop_analyze — Progressive Refinement

crop_analyze(
    image_source="screenshot.png",
    x=0.15, y=0.02, width=0.6, height=0.07,  # normalized 0.0-1.0
    depth=1,       # agent-managed, max 3
    task="ui"
)

Use coordinates from analyze_ui's suggested_regions. Max depth 3 prevents infinite recursion.

capture_page — Screenshot Fallback

capture_page(
    url="http://localhost:3000",
    viewport_width=1280,
    viewport_height=720,
    full_page=False,
    wait_for="networkidle"  # networkidle/domcontentload/load/none
)

Note: If your agent has built-in browser capability (Computer Use, Trae browser, Playwright MCP), use that instead and pass the screenshot path to analyze_ui.

Response Format

All tools return a structured JSON envelope:

{
  "status": "success",
  "tool": "analyze_ui",
  "result": { ... },
  "warnings": []
}

Status can be: success | partial | error | unavailable | unsupported | completed

Configuration

Variable	Default	Description
VISION_API_KEY	-	VLM API key (required)
VISION_BASE_URL	https://api.openai.com/v1	VLM API endpoint
VISION_MODEL_ID	gpt-4o	Model to use
VISION_TIMEOUT	60	Request timeout (seconds)
VISION_MAX_RETRIES	3	Max retry attempts
VISION_ALLOW_LOCAL_FILES	true	Allow local file paths
VISION_ALLOWED_PATHS	-	Allowed directories (comma-separated)
VISION_BLOCK_PRIVATE_IPS	true	Block SSRF to private IPs
VISION_MAX_IMAGE_SIZE_MB	10	Max image size
VISION_MAX_IMAGE_PIXELS	40000000	Max pixel count

Security

• SSRF Protection: Private IP addresses are blocked by default (VISION_BLOCK_PRIVATE_IPS)
• Path Restriction: Local file access can be restricted to allowed directories (VISION_ALLOWED_PATHS)
• Image Limits: Max image size and pixel count prevent resource exhaustion

Development

pip install -e ".[dev]"
pytest tests/

License

MIT