keyless-web-search-mcp

Standalone MCP stdio server: two keyless tools — web_search over a self-operated engine pool (Bing, 360, Baidu, Google, Naver, Yandex, DuckDuckGo) and web_fetch (anonymous HTTP(S) page reader). Each search keeps at most two usable engines, routes default priorities by query language, back-fills failed or irrelevant engines within a bounded attempt budget, relevance-ranks results, merges them, and dedupes by canonical URL. Self-contained directory, zero build step, not part of the harness package system — move it anywhere.

Why this exists

The harness's built-in web_search (DeepSeek provider) needs DEEPSEEK_API_KEY and sends every query to the DeepSeek cloud. This server is the local-model-friendly alternative: no keys, no vendor search API, works on mainland-China and HK networks (Bing/360/Baidu from mainland lines, Bing/360/Naver/Yandex from the tested HK line; DuckDuckGo is not reachable from any of them).

web_fetch exists for the same reason on the read side: the web profile mounts a search provider but no fetch provider, so the built-in web_fetch tool (even when a preset enables it) fails every call with WEB_PROVIDER_UNAVAILABLE. This tool reads full page content through the same keyless server instead.

Run

npm install --cache ./.npm-cache   # deps: @modelcontextprotocol/sdk, zod
node index.js                      # speaks MCP over stdio
npm test                            # run deterministic ranking/fallback tests

Quick probe without a client:

printf '%s\n' \
  '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"probe","version":"0.0.1"}}}' \
  '{"jsonrpc":"2.0","method":"notifications/initialized"}' \
  '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' \
  | node index.js

Install in Claude Code

Install it for the current user so every Claude Code project can use it. Resolve both paths when registering the server; this avoids depending on the shell's working directory or PATH later:

claude mcp add --transport stdio --scope user web-search-self -- "$(command -v node)" "/absolute/path/to/keyless-web-search-mcp/index.js"
claude mcp get web-search-self

Use --scope local instead to keep it private to the current project, or --scope project to write a shareable .mcp.json. Claude Code stores user/local registrations in ~/.claude.json; MCP server definitions do not belong in settings.json.

After registration, claude mcp list should report web-search-self as connected. The tools appear as mcp__web-search-self__web_search and mcp__web-search-self__web_fetch.

Tools

web_search

web_search({ query, count?, maxSources?, engines? })

Param	Default	Meaning
query	—	Search query, any language
count	8	Max merged results (1–20)
maxSources	2	Max usable engines per search (hard cap 2; fallback attempts are bounded)
engines	query-language dependent	Optional explicit pool in priority order; when omitted, Chinese/Korean/Japanese/Russian queries use region-appropriate priorities

Two-source policy. The pool is routed by query language when engines is omitted: Chinese starts with 360/Baidu/Bing; English starts with Bing/Google/Naver/360 so the four-attempt budget retains working fallbacks on the tested mainland/HK networks. An explicit engines list remains authoritative. Candidates run in bounded parallel rounds; an engine consumes a source slot only after it returns parseable results that pass conservative lexical relevance filtering. Failed, challenged, empty, or irrelevant engines are back-filled up to the attempt and collector budgets, and diagnostics are shown in an "Engine note" line. Results are ranked by query overlap with a small source-quality adjustment: obvious repost/content-farm signals are penalized, while official/docs/education/GitHub signals are favored without hard-blocking ordinary sites. Merging remains round-robin interleaved (engine A's 1st, engine B's 1st, A's 2nd, …), and canonical URL dedupe removes fragments, common tracking parameters, and safe www. differences.

Result: numbered list of title [engine] / real destination URL / snippet. Link cleanup per engine: Bing click-tracker links decode locally (base64 u parameter); 360 reads the data-mdurl attribute; Baidu reads the block's mu attribute (direct URL, with one best-effort redirect GET only for legacy link?url= wrappers); Naver and Yandex titles carry the direct URL in the anchor; Google /url?q= wrappers unwrap; DuckDuckGo /l/?uddg= trackers unwrap. Per-engine probe budgets: 10 s (Bing, Baidu), 8 s (360, Naver, Yandex), 5 s (Google, DuckDuckGo) — all run in parallel, so the round is only as slow as its slowest member.

web_fetch

web_fetch({ url, maxChars? })

Param	Default	Meaning
url	—	Absolute http(s) URL (anything else is rejected with a diagnostic)
maxChars	20000	Max content characters returned (1000–100000)

Anonymous public-web read, no credentials: browser User-Agent, at most five redirects, 20 s wall clock covering headers and body, response body cut at exactly 5 MB. Every initial and redirect target is checked after DNS resolution; loopback, private, link-local, carrier-grade NAT, reserved, multicast, and local hostnames are rejected to prevent SSRF into the machine or cloud metadata services. HTML is stripped to visible text (script/style/noscript/svg/head/iframe/canvas/form dropped, block boundaries become newlines, the <title> is carried in the header); textual, JSON, and XML media types pass through entity-decoded using their declared charset, while binary media types are rejected. Output is a status header — status, final url (after redirects), content-type, truncation — followed by the content. Non-2xx answers return the body head plus a note (403/429 read as bot-check or paywall), never a fabricated page.

Engines and their status

Verified 2026-07 from three lines (an earlier mainland line, a Shanghai Telecom line, and a Hong Kong Zenlayer datacenter line):

Engine	Endpoint	Status from these networks
bing	www.bing.com/search	✅ ~10 organic blocks on both lines. Depending on the line, www serves the SERP directly or 302s to the cn.bing.com twin (the engine follows it); cn uses direct result URLs instead of /ck/a trackers — both shapes parse
360	www.so.com/s	✅ clean 200; organic blocks carry the real URL in data-mdurl
google	www.google.com/search	Reachable (200) but this client/IP is not trusted for plain-HTML SERPs: the body is a no-JS enablejs meta-refresh wall with zero organic results. Detected and skipped with a diagnostic; from a cleaner IP it would join the merge (see the bypass attempts below)
baidu	www.baidu.com/s	✅ on the Shanghai and HK lines: the risk control gates script-shaped requests (UA-only 302s to the wappass.baidu.com image captcha regardless of cookies) but accepts the full document-navigation header signature (Accept + Referer + sec-fetch-* + Upgrade-Insecure-Requests) — stable across repeated runs, with or without cookies. The engine sends that signature; lines that gate harder (the earlier mainland line) still hit the detected-and-skipped captcha diagnostic. Parser reads the block's mu attribute (direct URL); non-organic cards (recommend lists, hot boards, related search) are skipped like ad slots elsewhere
naver	search.naver.com/search.naver	✅ on the HK line (200, ~10 fds-web-doc-root organic blocks, stable across runs; answers Chinese queries with Chinese results). Title anchor carries the direct URL; the Korean a11y label "새 창 열림" is stripped from titles and snippets. Korean-first index that also covers global technical sites; untested from mainland lines (would degrade to the standard unreachable diagnostic)
yandex	yandex.com/search	✅ on the HK line: same navigation-signature lesson as Baidu — script-shaped requests get the SmartCaptcha "not a robot" checkbox, but the full document-navigation header set plus the homepage's cookie session (yandexuid et al.) is served the plain HTML SERP (~50 Organic blocks, direct URLs in OrganicTitle anchors). The engine warms the session in-process (30-min TTL, one re-warm retry on a challenged answer). Caveat: from datacenter IPs Yandex runs a rolling rate state — a burst of searches flips the IP back into captcha mode for a few minutes, which is why it sits sixth in the pool (consulted only when the engines ahead of it fail)
duckduckgo	html.duckduckgo.com/html	❌ TCP-unreachable on all three tested lines (including HK); last-resort engine for networks where it works

Deliberately not in the pool: Sogou — 302s to sogou.com/antispider/, the same IP-wall class Baidu covers. Also scanned and rejected from the HK line: Mojeek (serves a captcha page), Ecosia (403 "Ecosia Firewall"), MetaGer (redirects to a non-result page), and Yahoo/Brave/Qwant/Startpage/goo.ne.jp (TCP-unreachable from both mainland and HK lines).

Google wall: what was tried

The 200 response from this IP is not a block but a JS challenge interstitial (~90 KB of obfuscated/encrypted JavaScript; the no-JS path is a meta-refresh into a dead end). The wall turns out to be two layers:

1. JS challenge (computational) — decoded, it computes a proof value and sets an SG_SS cookie (5-minute expiry), then reloads. This layer is solvable outside a browser: running the page's scripts in plain Node.js with a light DOM shim (cookie jar, navigator, Image, document) completed the computation, and the resulting SG_SS cookie was accepted once — Google answered 200 and issued the NID/AEC trust cookies it otherwise never sends to this IP.
2. Session-pattern layer (behavioral) — subsequent plain-HTTP follow-ups, including an exact replica of the script's own reload flow (emsg=SG_REL + matching sei, jar-merged cookies, browser headers), escalate to the google.com/sorry 429 anomaly wall. This layer judges the whole session (TLS/HTTP2 fingerprint, request pacing, method mix), which Node's OpenSSL/undici stack does not match.

Client-side levers that were tested and did not change layer 1's answer: the gbv=1 parameter, cookie warm-up, full browser fingerprint headers, the retry/enablejs flow, the /m entry, text-browser/feature-phone/IE6/old-Android UAs, a spoofed Googlebot UA (Google validates it), CONSENT cookies, POST submission (405), alternate TLDs, and the "click here" emsg=SG_REL link without the cookie. Google-proxy alternatives were also unreachable or walled from this network: Startpage and Qwant time out, Mojeek serves a captcha page, Ecosia 403s.

Conclusion: layer 1 is Node-solvable (demonstrated); layer 2 is not, from a non-browser network stack. The only reliable paths to Google from this machine are a clean-IP proxy or a real browser context (headless Chromium); both sit in the same IP/traffic-pattern layer that a scraper cannot honestly out-argue, and repeated probing risks extending the IP's anomaly window — so this server does not attempt them. Google stays a back-fill engine that activates on networks where it will serve a plain-HTML SERP.

Mount into DeepSeek Harness

Add to your cordis.yml (the MCP bridge hot-reloads this entry):

- id: mcp-search
  name: '@deepseek-ai/dsh-mcp-client'
  config:
    serverName: search
    transport: stdio
    command: node
    args: ['/absolute/path/to/keyless-web-search-mcp/index.js']

The model then sees the tools as mcp__search__web_search and mcp__search__web_fetch.

Limitations

• Search quality is heuristic: lexical relevance handles obvious off-topic results and Chinese n-grams; internal search-engine pages are excluded; content-farm/repost signals only adjust ordering, not a universal trust judgment. A relevant result is not proof that its claims are correct — fetch and cross-check important facts.
• Scraping, not an API: layout changes can break a parser; the failure is loud ("no parseable organic results"), never fabricated. The Google and DuckDuckGo parsers are written from documented SERP structure and are not live-verifiable from this network — tune them the first time their engines actually answer.
• Rate control: both public engines tolerate casual use; hammering draws bot challenges.
• Query leaves the machine for the consulted engines (that is the price of keyless search); with the two-source cap, at most two of them see a query per search.
• Fetch is a keyless public-page reader, not a browser or intranet client: JS-rendered content is invisible to it (same class of limit as the search parsers); pages that 403/429 anonymous clients are reported, not bypassed. Local/private/reserved network targets are deliberately blocked, including redirect destinations.