mcp-vision-server

English | 中文

MCP server that exposes an analyze_image tool: it forwards an image to a configurable OpenAI-compatible vision model and returns the model's text answer. Any MCP client — DSH through @deepseek-ai/dsh-mcp-client, Claude Desktop, or others — can call it.

Usage

Start the server over stdio:

export VISION_API_KEY=...
export VISION_MODEL=gpt-4o
npx -y mcp-vision-server

Pass --help to print the environment-variable reference. To run from a checkout instead of npm: pnpm install && pnpm build && node lib/bin.js.

The bin also reads a .env file placed next to it (copy .env.example to .env and fill it in); variables already present in the environment always win over .env values.

Connect from DSH

Add one @deepseek-ai/dsh-mcp-client instance to cordis.yml:

- id: mcp-vision
  name: '@deepseek-ai/dsh-mcp-client'
  config:
    serverName: vision
    transport: stdio
    command: npx
    args: ['-y', 'mcp-vision-server']
    env:
      VISION_API_BASE_URL: https://open.bigmodel.cn/api/paas/v4
      VISION_API_KEY: !!js process.env.VISION_API_KEY
      VISION_MODEL: glm-4v-plus

The model sees the tool as mcp__vision__analyze_image.

Connect from Claude Desktop

claude_desktop_config.json:

{
  "mcpServers": {
    "vision": {
      "command": "npx",
      "args": ["-y", "mcp-vision-server"],
      "env": {
        "VISION_API_KEY": "...",
        "VISION_MODEL": "gpt-4o"
      }
    }
  }
}

Tool: analyze_image

analyze_image(image, question?)

Argument	Required	Description
image	yes	Image source: http(s):// URL, local absolute path, or data:<mime>;base64,<data> URL
question	no	Question to answer about the image; defaults to VISION_DEFAULT_PROMPT

The tool returns the vision model's text answer. Failures return an isError result with a readable Chinese message instead of throwing. Image bytes travel only to the configured vision API and never enter the calling client's model context.

Configuration

Env var	Required	Default	Description
VISION_API_BASE_URL	no	https://api.openai.com/v1	OpenAI-compatible API root URL, without a trailing slash
VISION_API_KEY	yes	—	API key; startup fails loud when missing
VISION_MODEL	yes	—	Vision model name, e.g. gpt-4o, glm-4v-plus, qwen-vl-max
VISION_DEFAULT_PROMPT	no	请详细描述这张图片的内容。	Prompt used when the caller passes no question
VISION_MAX_IMAGE_BYTES	no	20971520	Upper bound on accepted image bytes
VISION_TIMEOUT_MS	no	120000	Vision API call timeout in milliseconds

Model Experience

analyze_image tool (served through the MCP client bridge)

What the model sees

The tool is served to MCP clients with the pinned name analyze_image, description, and input schema. The description is verbatim:

Verbatim tool description

分析一张图片：传入图片 URL、本地绝对路径或 data URL（base64），可附带问题；返回视觉模型的文字回答。

Input schema field descriptions

image: 图片来源：http(s) URL、本地绝对路径或 data:image/...;base64,...
question: 可选问题；缺省时默认要求描述图片

Token effect

Fixed schema cost while a client keeps the tool registered, plus the tool result text in each conversation that calls it. The image bytes themselves never enter the client's model context.

KV Cache effect

Each analyze_image call issues an independent model request to the configured vision API, separate from the client's own request prefix; the server's behavior never invalidates client-side reuse. The bridged result text grows the client conversation append-only.

Known Limitations and Deferred Work

• stdio only — no Streamable HTTP transport; the server is not reachable over the network.
• Single image per call — one image argument only; multi-image analysis is deferred.
• Tools only — the server exposes no MCP resources or prompts.
• No image transformation — bytes are passed through as received; large or exotic formats depend on the vision provider's tolerance.
• No URL allowlist — the server downloads whatever URL the caller passes, with no SSRF defense; keep it away from untrusted callers.

Development

pnpm install
pnpm test        # unit tests, 100% per-file coverage gate
pnpm test:e2e    # keyless stdio e2e against a stub vision API
pnpm build       # emit lib/

The e2e suite spawns the real bin over stdio and serves a stub OpenAI-compatible API over HTTP; no API key is needed.

Vision Bridge (paste images into text-only models)

vision-bridge.mjs is an OpenAI-compatible strip proxy (the same pattern as codex-vision-bridge) that gives text-only models image input: requests carrying image parts are answered by the vision MCP server, text-only requests pass through unchanged to the real text-only upstream.

# 1. Fill in the text-upstream key (only needed for text passthrough)
copy vision-bridge.env.example vision-bridge.env
# 2. Start the bridge
start-vision-bridge.cmd        # or: node vision-bridge.mjs
# 3. Point a provider at http://127.0.0.1:8787/v1 and declare the model
#    with input: [text, image] (e.g. a pi-ai provider entry in settings.yaml)

Fail-open: when the vision service fails, the request falls back to passthrough instead of breaking.