Headless Lead Scraping MCP Server v0.4.0

A UI-free MCP server for real-time public-web lead research with no search API provider, no Tavily key, and no hosted scraping service.

What it does

• Searches Google Maps first (browser-rendered) for local places — returns name, address, phone, and official website.
• Then searches web search engines (Yahoo, DuckDuckGo, Bing) directly at request time, with no search API key.
• Then searches related search engines (Startpage, browser-rendered) as an extra fallback source.
• Fetches target websites directly over HTTP.
• Falls back to headless Chromium/Playwright for JavaScript-rendered pages.
• Crawls relevant internal contact/about/team/social pages.
• Uses the configured LLM through the Vercel AI SDK for structured extraction and verification.
• Deduplicates results and filters by required fields.
• Stores leads locally in data/leads.jsonl.
• Filters stored leads and exports them as CSV or JSON files.
• Exposes everything as MCP tools over stdio.

Data-source priority

Sources are tried in order until enough results are found:

1. Google Maps (googlemaps) — browser-rendered google.com/maps/search/; best for local businesses/product companies (name, address, phone, website, rating).
2. Web search engines — Yahoo, DuckDuckGo, Bing (yahoo, duckduckgo, bing).
3. Related search engines — Startpage (startpage, browser-rendered) for additional coverage.

The order is configurable via SEARCH_ENGINES (comma-separated). Maps-related settings: MAPS_SEARCH_URL, MAPS_MAX_RESULTS.

No external data API

There is no Tavily, Bing Search API, Google Search API, SerpAPI, Apify, Bright Data, or hosted scraper in this version.

The server performs search by requesting normal public search-result webpages and Google Maps search pages directly. Engines that respond with a CAPTCHA or bot-challenge page are skipped automatically. This means there is no API account or API key for search, but a public search website is still necessarily involved: the open Internet has no universal built-in index that a standalone program can query without using some indexed search source or already-known URLs.

If a search website blocks automated requests, the server cannot and does not bypass CAPTCHA, login, paywalls, robots directives, rate limits, or access controls.

LLM

The only required AI dependency is your LLM endpoint. The server supports OpenAI-compatible endpoints, including:

• Ollama
• LM Studio
• OpenAI
• other OpenAI-compatible local/self-hosted endpoints

For maximum independence, use Ollama or LM Studio locally.

Playwright

Playwright is the browser engine, not a lead-data provider. It is used only when a normal HTTP fetch cannot obtain the rendered page content or when browser rendering is explicitly requested.

Install Chromium:

npm install
npx playwright install chromium

Install

cp .env.example .env
npm install
npx playwright install chromium
npm run build
npm start

For development:

npm run dev

Ollama

ollama pull qwen3

.env:

LLM_PROVIDER=ollama
LLM_MODEL=qwen3
LLM_BASE_URL=http://127.0.0.1:11434/v1
LLM_API_KEY=ollama

LM Studio

Start a local OpenAI-compatible server in LM Studio and set:

LLM_PROVIDER=lmstudio
LLM_MODEL=YOUR_LOADED_MODEL_ID
LLM_BASE_URL=http://127.0.0.1:1234/v1
LLM_API_KEY=lm-studio

MCP tools

• search_web — live web search in priority order (Google Maps, then web engines, then related engines); optional engines param.
• scrape_page — HTTP fetch + Playwright fallback.
• extract_leads — LLM extraction from supplied page content.
• find_leads — Google Maps place discovery first, then web + related engine search; crawls, extracts, verifies, deduplicates, filters by requiredFields, stores.
• list_stored_leads — read locally stored records.
• filter_leads — filter stored leads by keyword, field presence, and minimum confidence.
• export_leads — write stored leads (optionally filtered) to a CSV or JSON file on disk.
• clear_stored_leads — clear local records.

Example

{
  "query": "software development companies",
  "location": "California, USA",
  "limit": 25,
  "requiredFields": ["name", "website", "email"],
  "maxSearchQueries": 8
}

requiredFields filters out any lead missing all of the listed fields. Default is ["name"].

Export example:

{
  "format": "csv",
  "hasEmail": true,
  "minConfidence": "medium",
  "filename": "california-leads"
}

This writes data/california-leads.csv (or .json for format: "json") and returns the file path and row count.

Stored data

Default:

data/leads.jsonl

No hosted database is required. The file is local to the MCP server process.

Important limitation

A truly provider-free program cannot discover arbitrary Internet resources from nothing: search requires an index or known URLs. This project removes search APIs and hosted scraping providers and performs live search-page retrieval itself. It does not pretend that the Internet has a universal local index.

Only publicly accessible information is collected. The server does not bypass authentication, CAPTCHA, paywalls, robots/access controls, or other security mechanisms.