Yet Another WebFetch MCP Alternative

English | 中文

---

English

A local MCP server that fetches any http/https webpage, extracts the main content, and converts it to Markdown. Designed as a drop-in replacement for Claude Code's built-in WebFetch tool, which may fail under certain network / proxy environments (e.g., "Unable to verify if domain is safe").

Features

• Content extraction — Mozilla Readability for article pages; falls back to full-page conversion for non-article content.
• HTML → Markdown — powered by turndown; preserves headings and code blocks.
• Auto proxy detection — picks up Windows / macOS / Linux system proxy automatically (env vars also supported).
• Truncation — defaults to 50,000 characters to prevent context overflow.
• 20s timeout, redirect-follow, browser User-Agent.
• SSRF protection — http/https only; blocks localhost and private IP ranges (IPv4 & IPv6).

Installation

Requires Node.js ≥ 20.

git clone <repo-url> Yet-Another-WebFetch-MCP-Alternative
cd Yet-Another-WebFetch-MCP-Alternative
npm install

Setup with Claude Code

Add the following to ~/.claude.json under mcpServers (user-level) or to a project-local .mcp.json:

{
  "mcpServers": {
    "webfetch": {
      "type": "stdio",
      "command": "node",
      "args": ["<path-to-repo>\\index.js"]
    }
  }
}

Note: On Windows, avoid using npx as the command — it silently fails in non-TTY environments.

Reload the window (Ctrl+Shift+P → Developer: Reload Window), then type /mcp in a conversation to verify the connection.

Usage

The tool is registered as fetch_web.

Parameter	Required	Description
url	Yes	Full URL, http or https
maxChars	No	Max characters returned, default 50000
timeout	No	Timeout in milliseconds, default 20000

Claude will invoke the tool automatically when it needs to fetch a webpage — no manual prompting required.

Proxy

The server auto-detects the system proxy on all three platforms:

Platform	Detection method
Windows	Registry Internet Settings\ProxyServer
macOS	scutil --proxy
Linux	GNOME gsettings

To override manually, set the HTTPS_PROXY or HTTP_PROXY environment variable in the MCP config:

{
  "webfetch": {
    "type": "stdio",
    "command": "node",
    "args": ["<path-to-repo>\\index.js"],
    "env": {
      "HTTPS_PROXY": "http://127.0.0.1:7890"
    }
  }
}

Testing

npm test          # Handshake & tool list test (no network required)
LIVE=1 npm test   # End-to-end fetch against docs.tavily.com

Known Limitations

• No JavaScript rendering — dynamic pages only return the initial HTML (same as most scraping tools).
• Non-UTF-8 pages may produce garbled output in edge cases.
• Sites with strict anti-bot measures may return error pages or block requests.
• Domains unreachable from the local network (e.g., DNS-blocked in certain regions) cannot be fetched.

License

MIT

---

中文

一款本地 MCP 服务器：抓取任意 http/https 网页，提取正文并转换为 Markdown。作为 Claude Code 内置 WebFetch 工具的替代——内置 WebFetch 在部分网络 / 代理环境下会报 "Unable to verify if domain is safe" 而不可用。

特性

• 正文提取 — 基于 Mozilla Readability，非文章页面自动回退全文转换。
• HTML → Markdown — 基于 turndown，保留标题与代码块。
• 自动代理检测 — 自动读取 Windows / macOS / Linux 系统代理，也支持环境变量覆盖。
• 截断 — 默认 50000 字符，防止上下文溢出。
• 20s 超时、重定向跟随、浏览器 UA。
• SSRF 防护 — 仅 http/https，拒绝本机与内网地址（含 IPv6）。

安装

需要 Node.js ≥ 20。

git clone <仓库地址> Yet-Another-WebFetch-MCP-Alternative
cd Yet-Another-WebFetch-MCP-Alternative
npm install

接入 Claude Code

在 ~/.claude.json 的 mcpServers（用户全局）或项目根目录 .mcp.json 中加入：

{
  "mcpServers": {
    "webfetch": {
      "type": "stdio",
      "command": "node",
      "args": ["<仓库路径>\\index.js"]
    }
  }
}

注意： Windows 下不要用 npx 作为 command（无 TTY 环境下会静默失败）。

保存后重载窗口（Ctrl+Shift+P → Developer: Reload Window），在对话中输入 /mcp 确认连接状态。

使用

工具名为 fetch_web。

参数	必填	说明
url	是	完整 URL，http 或 https
maxChars	否	返回内容最大字符数，默认 50000
timeout	否	超时毫秒数，默认 20000

Claude 会在需要抓取网页时自动调用，无需手动引导。

代理

服务器自动检测三平台系统代理：

平台	检测方式
Windows	注册表 Internet Settings\ProxyServer
macOS	scutil --proxy
Linux	GNOME gsettings

如需手动覆盖，在 MCP 配置中设置 HTTPS_PROXY 或 HTTP_PROXY 环境变量：

{
  "webfetch": {
    "type": "stdio",
    "command": "node",
    "args": ["<仓库路径>\\index.js"],
    "env": {
      "HTTPS_PROXY": "http://127.0.0.1:7890"
    }
  }
}

测试

npm test          # 握手与工具清单测试（不依赖网络）
LIVE=1 npm test   # 额外做一次端到端抓取（docs.tavily.com）

已知限制

• 不渲染 JavaScript — 动态页面只能拿到初始 HTML（与多数抓取工具一致）。
• 非 UTF-8 编码页面在极端情况下可能乱码。
• 严格反爬站点可能被拦截或返回错误页。
• 本机网络 DNS 不可达的域名无法抓取。

许可证

MIT