crawlcheck-mcp-server

MCP server for crawlcheck. Crawls a website in a real Chromium browser and reports JavaScript console errors, uncaught exceptions, failed requests, broken links, and accessibility violations — classifying console errors and failed requests as first-party (your code) or third-party (embeds and scripts you don't control).

Runs locally over stdio, so crawls use your machine's browser and nobody pays for hosted compute.

Prerequisite

npx playwright install chromium

Without it the tools return: Chromium is not installed for Playwright. Run: npx playwright install chromium.

Configuration

Claude Desktop / Claude Code — .mcp.json:

{
  "mcpServers": {
    "crawlcheck": {
      "command": "npx",
      "args": ["-y", "crawlcheck-mcp-server"]
    }
  }
}

Cursor — same shape in mcp.json.

Tools

Tool	Purpose
crawlcheck_scan_site	Crawl a site. Returns a summary, the top 20 issues, and a scan_id. Never returns the full issue list.
crawlcheck_check_page	Check a single page without following links — the fast path for "is this page broken?".
crawlcheck_get_issues	Filtered, paginated issues from a completed scan. Reads from cache; never re-crawls, never blocks.

Typical flow: scan_site for the summary, then get_issues to drill in without re-crawling or flooding the context window.

Two behaviours worth knowing

Classification is not universal. Only console-error and failed-request carry a classification, because those are the only issue types for which the engine computes a resource origin. page-error, broken-link and a11y have no classification field at all — they are reported as unclassified, which does not mean third-party. crawlcheck_get_issues therefore defaults to classification: "all"; filtering to "first-party" would silently hide every broken link and accessibility violation.

One crawl at a time. A second concurrent scan is rejected immediately rather than queued:

A crawl is already running. crawlcheck runs one crawl at a time — wait for the current scan to finish, then retry. If you have a scan_id from an earlier scan, use crawlcheck_get_issues instead; it reads from cache and never blocks.

This is a correctness requirement, not throttling. runCrawl in crawlcheck 0.2.x redirects the global console.log while crawling (so that progress output can't corrupt this server's JSON-RPC frames on stdout), and overlapping calls would interleave that redirect. Rejecting beats queueing because MCP clients time out tool calls anyway, so a queued crawl usually dies waiting.

Exit semantics

summary.exitCode mirrors crawlcheck's process exit code and is only ever 0 or 1. Accessibility violations are advisory and do not affect it unless strict: true. Third-party console errors and failed requests are excluded unless include_third_party: true.

Development

npm install        # crawlcheck resolves to ../crawlcheck until 0.2.0 is on npm
npm run build      # generates src/types/report.v1.d.ts, then compiles
npm test           # builds, then runs the unit tests
npm run check:types  # regenerate types and fail if they drift from the schema
npm run inspect    # MCP Inspector against the built server

src/types/report.v1.d.ts is generated from crawlcheck's published JSON Schema and committed. Don't hand-edit it — npm run check:types regenerates and fails on any diff, so a crawlcheck schema bump can't slip through unnoticed. The only hand-written declaration is runCrawl's signature in src/types/crawlcheck.d.ts, which deliberately omits crawlcheck's internal raw field so that reaching for it is a compile error.

Verifying

Unit tests cover ranking, digest limits, cache TTL/LRU, single-flight (including that a failed crawl releases the lock), and error mapping. What they can't cover is a real crawl — for that:

npm run build
npx @modelcontextprotocol/inspector node dist/index.js

In Inspector v2.x the server appears in a Servers list as Disconnected — flip the toggle beside it to start the process before the tools show up.

Concurrency is the one behaviour the Inspector can't exercise, because it sends a single tool call at a time. For that:

node test/live-concurrency.mjs [url]

It drives the built server over stdio and fires a second crawlcheck_scan_site while the first is still crawling. Expected: the second is rejected with "A crawl is already running", the first completes, and a third succeeds afterwards — proving the lock releases rather than wedging the server. Exits non-zero on failure. Excluded from npm test because it launches a browser and hits the network.

License

MIT