SDW_Search

SDW_Search is a provider-neutral Web Search MCP server and CLI. It combines live search, scholarly discovery, verified PDF downloads, URL normalization, multi-provider ranking, secure page fetching, caching, and citation-ready research evidence.

Current Tools

MCP tool	Purpose
web_search	Search configured providers and return normalized, deduplicated candidate sources
web_fetch	Fetch a public URL and extract readable text with redirect and SSRF checks
academic_search	Discover papers and download verified PDFs into reference, with manual fallback reasons
browser_auth	Start, inspect, or close the browser session used for authenticated PDF downloads
browser_open	Open a page or download target in the system default browser with its existing SSO session
research	Run multiple discovery queries, fetch diverse pages, and return evidence for client-side synthesis
search_status	Show provider configuration, cache state, version, proxy state, and safety limits

Search snippets are discovery data. research only promotes successfully fetched page text into evidence.

Requirements

• Node.js 20.18.1 or newer
• Google Chrome or Microsoft Edge for browser-authenticated downloads
• Network access, optionally through SDW_PROXY_URL
• No API key is required for the default Bing and DuckDuckGo providers

Install And Run

npm install
npm run build
node dist/cli.js status
node dist/cli.js search "OpenAI Responses API web search" --max-results 5
node dist/cli.js academic "metasurface antenna" --journal "IEEE Antennas and Wireless Propagation Letters" --max-papers 3
node dist/cli.js auth "https://ieeexplore.ieee.org" --auth-mode persistent
node dist/cli.js fetch "https://example.com"
node dist/cli.js open "https://ieeexplore.ieee.org/document/11536863/"
node dist/cli.js download "https://publisher.example/paper.pdf"
node dist/cli.js research "Compare two web search APIs" --depth standard

Start the stdio MCP server:

node dist/mcp.js

Generic MCP client configuration:

{
  "mcpServers": {
    "SDW_Search": {
      "command": "node",
      "args": ["F:/absolute/path/to/SDW_Search/dist/mcp.js"],
      "env": {
        "SDW_SEARCH_PROVIDERS": "auto"
      }
    }
  }
}

Use an absolute path because MCP clients may start the server from another working directory.

Providers

SDW_SEARCH_PROVIDERS accepts a comma-separated list:

• duckduckgo: keyless DuckDuckGo Lite search
• bing: keyless Bing HTML search with RSS fallback
• brave: official Brave Search API, configured by SDW_BRAVE_API_KEY
• searxng: one or more comma-separated instances in SDW_SEARXNG_URL
• auto: enables configured API providers plus Bing and DuckDuckGo

Example:

$env:SDW_SEARCH_PROVIDERS = "brave,bing,duckduckgo"
$env:SDW_BRAVE_API_KEY = "..."
node dist/cli.js search "query"

Configuration

Variable	Default	Meaning
SDW_SEARCH_PROVIDERS	auto	Enabled provider list
SDW_BRAVE_API_KEY	empty	Brave Search API key
SDW_SEARXNG_URL	empty	Comma-separated SearXNG base URLs
SDW_PROXY_URL	auto-detected	Explicit HTTP(S) proxy URL
SDW_HTTP_TIMEOUT_MS	15000	Per-request timeout
SDW_MAX_FETCH_BYTES	5242880	Maximum downloaded page size
SDW_MAX_PDF_BYTES	52428800	Maximum PDF download size
SDW_SEARCH_CACHE_TTL_SECONDS	900	Search cache lifetime
SDW_FETCH_CACHE_TTL_SECONDS	21600	Fetched page cache lifetime
SDW_CACHE_PATH	~/.sdw-search/cache.sqlite	SQLite cache path
SDW_ALLOW_PRIVATE_NETWORK	false	Allow fetching private network targets
SDW_BROWSER_HANDOFF_ENABLED	true	Allow explicit handoff of public URLs to the system default browser
SDW_AUTH_MODE	persistent	Authenticated download mode: none, persistent, cdp, or experimental extension
SDW_BROWSER_CHANNEL	chrome	Persistent browser channel: chrome, msedge, or bundled chromium
SDW_BROWSER_PROFILE_DIR	%LOCALAPPDATA%/SDW_Search/browser-profile	Dedicated persistent profile; never set this to the daily Chrome/Edge data directory
SDW_BROWSER_BRING_TO_FRONT	false	Keep automated browser navigation in the background; on Windows, persistent mode starts minimized
SDW_BROWSER_HEADLESS	false	Run the dedicated browser headless; keep false for interactive SSO/MFA
SDW_BROWSER_STARTUP_TIMEOUT_MS	30000	Browser launch, connection, and initial navigation timeout
SDW_CDP_ENDPOINT	empty	Explicit HTTP(S) or WebSocket CDP endpoint required by cdp mode
SDW_REFERENCE_DIR	<current directory>/reference	Academic PDF and per-run report directory

On Windows, the enabled system proxy is detected from Internet Settings when no proxy environment variable is present. NO_PROXY is honored. Keep SDW_ALLOW_PRIVATE_NETWORK=false for normal MCP use.

Institutional SSO

The default persistent mode uses a dedicated browser profile. The profile starts lazily, so ordinary searches do not open a browser. By default it starts minimized on Windows and automated navigation does not take foreground focus. Click its taskbar icon when manual SSO/MFA is required, or set SDW_BROWSER_BRING_TO_FRONT=true to restore foreground behavior.

1. Call browser_auth with action: "start", auth_mode: "persistent", and optionally an institution or publisher URL.
2. Complete the school SSO and MFA in the dedicated Chrome/Edge window.
3. Leave the MCP server running and call academic_search. Cookies and browser storage persist across later browser and MCP restarts. Authenticated HTTP retries use the profile's cookie jar; browser storage remains available to publisher pages and is never exported.

If academic_search discovers that the profile still needs authentication, it opens the paper page and returns manual_required with instructions to complete SSO and run the search again. SDW_Search does not return, log, or copy browser cookies. Treat the profile directory as sensitive credential storage and do not sync or commit it.

For CLI use, run sdw-search auth <url> first. It keeps the dedicated browser open until Ctrl+C so the interactive login can finish and the profile can close cleanly.

Set SDW_BROWSER_CHANNEL=msedge to use Edge. chromium requires the matching Playwright browser binary to be installed. Do not point SDW_BROWSER_PROFILE_DIR at the normal Chrome or Edge User Data directory; Playwright does not support automating the regular profile this way.

cdp is an advanced alternative for a browser that explicitly exposes a debugging endpoint:

& "$env:ProgramFiles\Google\Chrome\Application\chrome.exe" `
  --remote-debugging-port=9222 `
  --user-data-dir="$env:LOCALAPPDATA\SDW_Search\cdp-profile"
$env:SDW_AUTH_MODE = "cdp"
$env:SDW_CDP_ENDPOINT = "http://127.0.0.1:9222"

Chrome 136 and newer ignore command-line remote-debugging switches for the default Chrome data directory. SDW_Search only connects to the configured endpoint and disconnects without closing the attached browser. It does not yet auto-discover the Chrome 144+ chrome://inspect/#remote-debugging consent endpoint.

extension is an interactive mode backed by the pinned official Playwright MCP sidecar. Install the Playwright Extension in the Chrome/Edge profile you want to reuse, then call browser_auth with action: "start" and auth_mode: "extension". The browser asks you to approve a tab; an optional URL is opened in a new controlled tab. SDW_Search never exports browser cookies.

Extension mode can reuse the daily browser's selected tabs and login state. For supported publishers such as IEEE Xplore, SDW_Search performs a same-origin fetch inside the approved tab, relays the PDF in bounded chunks, verifies it, and writes it into reference. PDF bytes are removed from page memory after the relay, and sidecar snapshots/logs use a temporary directory that is deleted when the bridge closes. Unsupported publishers still return extension_download_path_uncontrolled with a manual URL. Leave PLAYWRIGHT_MCP_EXTENSION_TOKEN empty for interactive approval; if configured, treat it as a secret.

The existing browser_open tool remains available as a simple system-browser handoff. It can use the daily browser session, but SDW_Search cannot verify that download or its final path.

Academic Downloads

academic_search combines OpenAlex metadata with the configured web providers, filters obvious non-paper pages, resolves common PDF targets such as IEEE Xplore document pages and arXiv abstracts, and verifies the %PDF file signature before writing anything. It first tries public HTTP and only starts the configured browser authentication mode after an authentication or publisher-block response.

For IEEE Xplore browser fallback, persistent/CDP mode visibly opens the article and IEEE PDF wrapper, then captures the raw PDF response before Chrome's built-in PDF Viewer transforms it. Provider discovery, signature/hash verification, and the final reference write remain background operations.

The known IEEE persistent flow has an opt-in live regression test. Close any SDW_Search dedicated Chrome window first, then run this from PowerShell:

$env:SDW_RUN_IEEE_TESTS="1"
npx tsx --test tests/ieee-persistent.integration.test.ts
Remove-Item Env:SDW_RUN_IEEE_TESTS

The test uses the configured persistent profile, visibly opens IEEE Xplore, forces the public-request failure path, captures the raw PDF response, verifies the %PDF signature, and writes only to a temporary reference directory that is removed afterward.

The equivalent real CDP regression test expects an authenticated Chrome/Edge instance already running with a non-default --user-data-dir and the configured SDW_CDP_ENDPOINT:

$env:SDW_RUN_IEEE_CDP_TESTS="1"
npx tsx --test tests/ieee-cdp.integration.test.ts
Remove-Item Env:SDW_RUN_IEEE_CDP_TESTS

It also forces the public-request failure path, so a pass proves the PDF came through the attached browser session. The test disconnects from CDP without taking ownership of the external browser and removes its temporary download directory.

The Extension relay has a separate opt-in live regression test. Keep Chrome open with the official extension installed, run the following command, and approve the connection once:

$env:SDW_RUN_EXTENSION_TESTS="1"
npx tsx --test tests/ieee-extension.integration.test.ts
Remove-Item Env:SDW_RUN_EXTENSION_TESTS

The default output is the reference folder under the MCP process working directory. Set SDW_REFERENCE_DIR to an absolute path in the MCP configuration when the client uses an unpredictable working directory. Filenames are sanitized and include a stable URL hash; existing files are reported as already_exists and never overwritten. Every run also creates an academic-search-*.json report in the same folder.

Failed papers are returned as manual_required with a reason code and manualUrl. Typical reasons are authentication_required, publisher_blocked, browser_unavailable, extension_download_path_uncontrolled, not_pdf, network_error, and file_too_large. Successful results identify whether the file came from public_http, browser_authenticated, or an existing_file.

Ranking And Safety

Search processing includes:

1. Tracking-link decoding and URL canonicalization.
2. Cross-provider duplicate fusion using reciprocal-rank fusion.
3. Query relevance scoring.
4. MMR-style diversity selection to reduce same-domain repetition.

Page fetching allows only HTTP(S), rejects credentials and private/reserved destinations, resolves DNS before every request, validates every redirect, limits redirect count and response size, and labels returned page content as untrusted.

Development

npm run check
npm test
npm run build

Inspect the MCP server with any MCP inspector by launching node dist/mcp.js over stdio.