brightdata-gatekeeper

Evidence-based approval for Bright Data Scraper Studio self-healing.

Bright Data repairs broken scrapers. It does not decide whether a repair is correct. Gatekeeper does — it detects the break, writes the heal brief, then judges the proposed fix against rows recorded while the scraper was healthy, and rejects it if they disagree.

Dashboard	https://gatekeeper.rycerz.es — 29 collectors, 6,496 rows
Mutation target	https://recalls.rycerz.es
Walkthrough	docs/DEMO.md
Demo video	video/out/gatekeeper-demo.mp4 — 83s, generated from HTML, not screen-recorded

The gap this fills

Bright Data's own CLI documentation states it plainly:

You are the detector. The CLI never decides on its own that a scraper is broken — you inspect the run output and decide. […] Vague prompts produce vague heals.

scraper heal rewrites a collector's template and stops at an approval gate. approve commits it. --auto-approve skips the question. Nobody checks the answer.

Bright Data ships	Gatekeeper adds
scraper run	Detector — catches breakage that still returns HTTP 200
scraper heal "<prompt>"	Brief generator — a typed field list carrying the diagnosis
the gate, approve, --reject	Approver — judges the preview against golden rows

run → validate → diagnose → brief → heal → verify → approve or reject → re-brief

The cycle is also reachable from nowhere else. Bright Data's Python SDK covers Scraper Studio's trigger/poll/fetch but not heal or approve; its 69-tool MCP server has no Scraper Studio tools at all. Gatekeeper ships an MCP server, so an agent can drive it.

The result that matters

Against a live collector on a deliberately broken page, Bright Data's heal recovered four fields correctly and silently dropped the fifth to null. units is declared optional, so every schema, type and range check passed — zero broken findings:

--auto-approve would : APPROVE   preview passes every field check
gatekeeper decided   : REJECT    2/2 golden rows disagree
                                 units: expected '118500', preview has 'null'

--auto-approve would have committed a scraper that quietly stopped collecting unit counts. The rejection reason went into the next brief, the retry matched golden on every compared row, and the fix was verified on a fresh live run.

Quick start

uv sync --all-extras
npm install -g @brightdata/cli        # Node >= 20
cp .env.example .env                  # add BRIGHTDATA_API_KEY

gatekeeper doctor                     # check the setup
gatekeeper spec add specs/mutant-recalls.json
gatekeeper status                     # survey every collector
gatekeeper check   <collector_id>     # detect
gatekeeper golden  <collector_id>     # record known-good rows
gatekeeper heal    <collector_id>     # run the loop
gatekeeper history <collector_id>     # why a fix was committed or rejected
gatekeeper serve                      # dashboard on :8000

As an MCP server, so an agent can drive the loop:

claude mcp add gatekeeper -- gatekeeper mcp

preview_heal_brief shows the prompt that would be sent without spending an AI-Flow job.

Rules the loop obeys

• Never trust an exit code. A run returning 200 with empty fields is the failure that matters.
• Never heal a block. blocked and dead_page alarm but are not layout changes.
• Never approve without evidence. No golden rows means abstain, not approve.
• Verify after committing. A preview is a promise; a live run is evidence.

Collectors

Four managed collectors, three of them real sites, none in Bright Data's 1,743 pre-built scrapers — which gatekeeper coverage <url> checks rather than asserts.

Collector	Target	Rows	Notable
mutant-recalls	recalls.rycerz.es	8	Breaks on demand; five layouts, identical values
arxiv-cs-lg	arxiv.org	950	Stable identity, so golden rows survive churn
hn-front-page	news.ycombinator.com	59	Nested output, flattened via row_path
lobsters	lobste.rs	125	Arrived broken: 114 rows, every field absent

Details, including the golden-bootstrap problem: docs/SITES.md.

Beyond those four, the loop was swept across 50 further sites — one collector each, spec inferred from its own output rather than hand-written. 20 diagnosed healthy across 5,354 records, and the sweep found four real defects in Gatekeeper that single-site testing never exposed, including rate-limited runs reading as healthy. Matrix and failure taxonomy: docs/SWEEP.md.

Layout

src/gatekeeper/
  detector/    rules, baselines, engine — decides a scraper broke
  brief/       diagnosis → typed heal prompt
  approver/    judges a preview against golden rows
  brightdata/  bdata CLI wrapper, envelopes, coverage check
  controller   the closed loop
  mcp          agent-callable tools
  api, static  the dashboard
mutant/        the mutation target, deployed as a Worker

Tests

$ pytest -q
161 passed

No network and no credits: the bdata CLI is stubbed at the subprocess boundary and the loop runs against a scripted client, so the whole state machine — including reject-and-reconverge — is verified offline.

Notes

Deployment, and how the dashboard is published without credentials: docs/DEPLOY.md. Things the live API taught us that the docs did not: docs/FINDINGS.md.

MIT.