raw-web-search

An experimental MCP search server for Claude Code. Returns raw, verbatim web content instead of AI-rewritten summaries.

This is a research project, not a production tool. It explores whether raw web extraction can replace Claude Code's built-in WebSearch + WebFetch. The answer: it depends on the use case. See Eval Results for honest benchmarks.

What it does

One MCP tool call that searches the web, fetches pages, extracts content, and returns the most query-relevant excerpts — all verbatim from the source.

search("how to implement rate limiting in FastAPI")
→ 5 results with verbatim excerpts from real pages (~1,400 tokens total)

Also includes extract(urls) for direct URL content extraction (replaces WebFetch).

When to use this instead of WebSearch

Use case	raw-web-search	WebSearch	Winner
Code examples / API docs	Verbatim code preserved	Paraphrased by Haiku	raw-web-search
Batch searching (20+ searches/session)	Auto-approvable MCP tool	Click-per-call approval	raw-web-search
URL extraction (replacing WebFetch)	~100% success	~50% success	raw-web-search
General knowledge questions	72% factual coverage	96% factual coverage	WebSearch
Speed	4.5s	~3s	WebSearch
Token efficiency	~1,400 tok	~700 tok	WebSearch

Bottom line: Use raw-web-search when you need verbatim source text (code, docs, exact quotes) or want frictionless auto-approved searching. Use WebSearch when you need synthesized overviews or general knowledge.

Eval Results

Quantitative (20 queries)

Metric	raw-web-search	WebSearch	WebSearch+WebFetch
Latency	4.5s avg (2.7-8.1s)	~3s	~6-10s (multi-call)
Tokens/query	~1,400	~700	~1,400+
Results per query	5 (zero failures)	10 links + summary	1-3 per WebFetch
Tool calls	1	1	3-6
Content type	Verbatim excerpts	AI-rewritten	AI-summarized
Extraction success	100%	N/A	~50%
Auto-approvable	Yes	No	No

Quality (5 queries, 25 ground-truth facts)

Eval	Metric	raw-web-search	WebSearch
Factual coverage	Facts found in output	18/25 (72%)	24/25 (96%)
Technical queries	Direct comparison	Competitive	Good
Science/explainer	Direct comparison	Behind	Better
Downstream accuracy	LLM answer quality	0 wins, 2 ties, 1 loss	1 win, 2 ties

WebSearch wins on factual coverage because it uses an LLM (Haiku) to synthesize across 10+ full pages. We use embeddings to select the most relevant paragraphs from 5 pages — a fundamentally different (and less thorough) approach.

What we learned

1. AI synthesis > embedding selection for factual coverage. An LLM reading full pages and picking key facts will always beat cosine-similarity paragraph selection.
2. Verbatim text > AI summaries for code. Paraphrased code examples and API signatures are useless. Raw extraction preserves them.
3. Auto-approval is the killer feature. The biggest real-world advantage isn't content quality — it's eliminating click-per-search friction.
4. WebFetch is genuinely bad. 50% failure rate on real-world URLs. Our extract tool is a clear improvement.

How it works

Query
  → SearXNG (search 70+ engines via Docker)
  → Fetch HTML (httpx concurrent, 4s timeout)
  → Playwright fallback for failed URLs (JS-rendered pages)
  → Extract content (trafilatura, F1=0.958)
  → Split into paragraphs
  → Embed query + paragraphs (fastembed, all-MiniLM-L6-v2)
  → Select top chunks by cosine similarity (~1500 chars/result)
  → BM25 rank results
  → Return structured text

Prerequisites

• Python 3.10+
• Docker (for SearXNG search backend)

How Docker is used

SearXNG runs as a Docker container alongside Redis. The MCP server auto-starts containers on demand.

• Auto-start: Checks if SearXNG is reachable, runs docker compose up -d if not
• Port: localhost:8888 (configurable via SEARXNG_URL)
• Stopping: docker compose down in the project directory

Install

# Install globally (available in all projects)
claude mcp add -s user open-search -- uvx --from git+https://github.com/andresfortunato/raw-web-search.git open-search-mcp

# Set as default search (one-time — tells Claude to prefer this over WebSearch)
uvx --from git+https://github.com/andresfortunato/raw-web-search.git open-search-mcp --setup

# Optional: install Chromium for ~100% extraction success (vs 76% without)
playwright install chromium

Tools

search

Search the web and return extracted content.

search(query, max_results=5, include_domains=None, exclude_domains=None, time_range=None)

extract

Extract content from specific URLs. Direct replacement for WebFetch.

extract(urls, query=None)

Configuration

Variable	Default	Description
SEARXNG_URL	http://localhost:8888	SearXNG instance URL
FETCH_TIMEOUT	4	HTTP fetch timeout in seconds
OPEN_SEARCH_CHUNK_CHARS	1500	Target chars per result
OPEN_SEARCH_DEBUG	false	Log pipeline timing per query

Architecture

src/open_search_mcp/
  server.py     # MCP server, tool definitions, Docker lifecycle
  searcher.py   # SearXNG client + BM25 scoring
  extractor.py  # URL fetching + trafilatura + Playwright fallback
  chunker.py    # Embeddings-based chunk selection (fastembed)
  cache.py      # TTL-based URL cache

Research

The research/ directory contains all eval data, benchmarks, and analysis:

• eval_a_v2.md — Factual coverage scoring (72% vs WebSearch's 96%)
• quality_eval.md — Direct content comparison (Eval C)
• baseline_comparison.md — Full baseline eval write-up
• eval_20_results.json — 20-query benchmark data

License

MIT