websearch-mcp

An MCP server that hooks into Claude Code and answers queries with full page content, not just snippets.

Flow when Claude calls the websearch tool:

Claude: websearch("PHP send email class")
  → 1. SearXNG search   → top N result URLs (title + url + snippet)
  → 2. BrowserOS        → open each URL as a background tab, read body text (no HTML)
  → 3. fallback         → Playwright (headless Chromium) if present, else plain HTTP fetch
  → 4. return           → [{title, url, text}] back to Claude

"10 tabs at a time": BrowserOS opens up to WEBSEARCH_RESULTS background tabs in one batch and reads them in parallel. Any page that fails (blocked, DNS, 404) is skipped gracefully and reported, with a per-page fallback to Playwright/curl.

Setup

npm install
npm run build

Node 18+. Playwright is optional — if globally installed (npm root -g) the fallback picks it up automatically; otherwise it silently uses plain HTTP fetch.

Register in Claude Code

In the project you want to use it from, copy the template and set your absolute path:

cp .mcp.json.example .mcp.json   # then edit <YOUR_CLONE_DIR>

{
  "mcpServers": {
    "websearch": {
      "command": "node",
      "args": ["<YOUR_CLONE_DIR>/dist/index.js"],
      "env": { "WEBSEARCH_SEARXNG_URL": "http://100.77.7.3:8890/search", "WEBSEARCH_BROWSEROS_URL": "http://127.0.0.1:9002/mcp" }
    }
  }
}

.mcp.json holds a machine-specific absolute path, so it's gitignored here — don't commit it. Configure per-machine from the .mcp.json.example template.

Then in a Claude Code session ask anything, e.g. "Search PHP send email class and summarize the top 3 results."

Config (env vars)

All config is via env vars in the MCP server entry — no code changes to move endpoints.

Var	Default	Meaning
WEBSEARCH_SEARXNG_URL	http://100.77.7.3:8890/search	SearXNG endpoint (JSON format auto-appended)
WEBSEARCH_BROWSEROS_URL	http://127.0.0.1:9002/mcp	BrowserOS MCP (streamable HTTP)
WEBSEARCH_RESULTS	10	Top N results to fetch full text for (max 12)
WEBSEARCH_MAX_PAGE_CHARS	8000	Truncate each page's text to this many chars (context budget)
WEBSEARCH_PAGE_TIMEOUT_MS	25000	Per-page read/navigation timeout
WEBSEARCH_USE_BROWSEROS	1	Use BrowserOS batch extraction
WEBSEARCH_USE_PLAYWRIGHT	1	Fall back to headless Chromium
WEBSEARCH_USE_CURL	1	Last fallback: plain HTTP fetch

Set WEBSEARCH_USE_BROWSEROS=0 to force the Playwright/curl path.

Tool

• websearch(query, n?) — query string, n = how many top results to read (default WEBSEARCH_RESULTS). Returns JSON: { query, results: [{ title, url, text, source, error? }] }. source is browseros / playwright / curl; error present on failures (e.g. bot-blocked).

Layout

src/
  index.ts               MCP stdio server + websearch tool
  config.ts              env-var config
  searxng.ts             SearXNG JSON search
  text.ts                html-strip / squash / bot-block heuristics
  extraction/
    extractor.ts         orchestrates BrowserOS → playwright → curl
    browseros.ts         MCP *client* to BrowserOS (tabs new/list, read, close)
    playwright.ts        headless Chromium fallback (auto-resolves global install)
    curl.ts              realistic-browser HTTP fetch + cheerio text extraction