vision-mcp

An MCP server that gives conversational image understanding to text-only models (DeepSeek, Claude Code, etc.).

The main model hands image paths and a question to a vision model, which returns a text description that the main model can reason over. A session mechanism enables "blind men and the elephant" style follow-ups: you can dig deeper into the same batch of images across multiple turns, and the vision model re-sees the full images and the conversation history on every turn.

Features

• Multi-turn conversational follow-up: follow-ups within a session automatically carry history context, supporting referential questions ("What does that sign say?")
• OpenAI-compatible vision API: any compatible endpoint works (default SiliconFlow; Qwen3.5-35B-A3B verified to accept images)
• Path deduplication: paths passed on session reuse are compared against existing ones; only new images are added
• URL support: pass http/https image URLs directly; they are forwarded to the vision API as-is (no local download)
• Image integrity validation: checks extension vs. actual format consistency; supports png/jpg/jpeg/webp/gif/bmp/tif/tiff
• Concurrency safe: operations on the same session are serialized; atomic writes; deletion is mutually exclusive with in-flight requests, so a deleted session can never be resurrected by a stale save
• Auto-expiry: sessions idle for 24 hours are cleaned up (configurable)
• Passthrough by default: images are sent as-is — no compression, no scaling, no re-encoding — unless compression is enabled (see max_image_mb)
• Staged compression (optional): when enabled, images over the threshold are compressed in stages, format-preserving where possible (JPEG/WebP lower quality first, then downscale; PNG keeps transparency by downscaling before falling back to JPEG)

Installation

git clone https://github.com/whyneedai/vision-mcp.git
cd vision-mcp
python3 -m venv .venv
.venv/bin/pip install -r requirements.txt

Configuration

The config file lives at ~/.config/vision-mcp/config.json:

{
  "vision_model": {
    "base_url": "https://api.siliconflow.cn/v1",
    "api_key": "{env:SILICONFLOW_API_KEY}",
    "model": "Qwen/Qwen3.5-35B-A3B",
    "enable_thinking": false,
    "max_tokens": 131072,
    "temperature": 0.1
  },
  "max_history_rounds": 4,
  "sessions_dir": "~/.local/share/vision-mcp/sessions",
  "session_ttl_hours": 24,
  "system_prompt": "optional, overrides the built-in vision system prompt"
}

Field	Description
max_history_rounds	Number of recent Q&A rounds (1 round = one question + one answer) sent to the vision model as context. History beyond the window is kept on disk but not sent. Default 4; 0 sends no history. Capped by the provider's message limit (10)
vision_model.base_url / api_key / model	OpenAI-compatible endpoint; {env:XXX} references an environment variable
vision_model.enable_thinking	Disable thinking mode (otherwise the API returns an empty content)
vision_model.max_image_mb	Compression threshold in MB: images at or above this size are auto-compressed below it (staged, format-preserving). Unset / empty / 0 disables compression (default)
sessions_dir	Session storage directory (default ~/.local/share/vision-mcp/sessions)
session_ttl_hours	Session idle-expiry in hours (default 24)
system_prompt	Vision system prompt (default: strictly follow the question, no hallucination)

Connecting to opencode

Add to the mcp section of your opencode config:

{
  "mcp": {
    "vision": {
      "type": "local",
      "command": ["/path/to/vision-mcp/.venv/bin/python", "/path/to/vision-mcp/server.py"],
      "environment": {
        "SILICONFLOW_API_KEY": "{env:SILICONFLOW_API_KEY}"
      },
      "enabled": true,
      "timeout": 300000
    }
  }
}

Tools

ask_image

Ask a question about one or more images, with multi-turn follow-up support.

Parameter	Required	Description
question	yes	The question
session_id	no	Existing session ID; omit to create a new session (image_path is then required)
image_path	no	List of image paths or http/https image URLs; may be omitted on session reuse (existing images are kept), new entries are deduplicated and appended. URLs are forwarded as-is to the vision API

Returns {session_id, answer, image_paths}. Relative paths resolve against the opencode working directory.

end_session

Delete a session and all of its related files. Original images are never deleted.

Architecture

server.py   MCP entry point: component wiring + tool registration
config.py   Config loading ({env:XXX} resolution)
sessions.py Session storage: atomic writes, per-session locks, TTL cleanup
images.py   Path/content validation, staged compression, data URL encoding
vision.py   Vision client: OpenAI-compatible API

Tests

.venv/bin/python test/test_mcp_proto.py   # MCP handshake and tool registration
.venv/bin/python test/test_e2e.py         # end-to-end (requires a real API key)

License

MIT