docverify-mcp

An MCP server that exposes document extraction and its deterministic verification as tools, so an assistant is structurally unable to report a number that failed its own tie-out check.

It reads an Ohio black-box test of State cash-basis "Combined Statement of Receipts, Disbursements and Changes in Fund Balances" (a public record), re-derives every total from the underlying line items, and only then decides whether it has an answer to give.

The property

The model cannot report a number this server has not proven, because the server does not give it one.

That is a property of the server, not of one tool, and it is a shape rather than a flag. extract_statement returns one of two different objects:

outcome	shape
verified	verified: true, and a line_items key holding the figures
refused	verified: false, and no line_items key at all, at any depth

Not line_items: null. Absent. A caller that reaches for the figures on a refusal gets a KeyError, not a number.

When a document fails, every tool withholds it — not just the one you asked. Figures still appear in a refusal, but only ever as operands of a check that did NOT tie out, which is the failure diagnosis the spec requires. The operands of checks that passed are withheld too: a check that passes inside a failed document proves internal consistency of one subset, it does not prove the figure. That holds on the cold path as well, because the verdict is computed wherever a document is loaded rather than at one call site, and the session carries the verdict rather than just the document.

Tested in tests/test_disclosure.py, which states the property and sweeps it across all three tools, every check id in the document, and both the cold and post-refusal paths; in tests/test_contract_shape.py, which walks the payload recursively looking for any figures field; and in tests/test_mcp_stdio.py, which asserts the same thing about what actually crosses the wire.

Tools

tool	returns
extract_statement(path)	the structured statement plus verified and the check summary, or a refusal with the failing checks and no figures. Takes a report PDF
verify_totals(path | statement)	every foot / crossfoot / articulate check, with the arithmetic shown for everything the server is entitled to disclose
explain_exception(check_id)	which two numbers disagreed, by how much, and the page each came from

extract_statement takes a report PDF, not a pre-extracted .json: verified: true means this server extracted these figures from a document and proved them, which is a different claim from you handed me numbers and they are internally consistent. verify_totals does accept caller-supplied numbers, because that is honestly what it is — check my arithmetic. Its payload reports checks_pass, never verified: true, and carries claim.kind == "arithmetic_only" saying in words that the figures were not extracted by this server.

Three check kinds, all re-derived from the line items:

• FOOT: the detail lines in a section sum to that section's reported total, per column.
• CROSSFOOT: for every row, the fund-type columns sum to the Combined Total column.
• ARTICULATE: net change = receipts - disbursements; ending = beginning + net change.

Each check ties out to within $1 (tolerance 1.0), which absorbs the rounding a published statement already carries and nothing larger.

On the bundled sample report's FY2024 statement that is 25 checks, all passing, relating all 39 (line item, column) cells of the 13 line items to something other than themselves. 25 is not observed, it is required: it is what the document's shape says must run, and fewer is a refusal: the manifest is derived from the document's structure, and a run producing fewer checks than the manifest refuses instead of reporting a pass. Zero checks is never a pass: a document the server could not derive a manifest from is refused, not reported as verified with nothing to show.

That report also contains the prior-year comparative on page 17, which is a different statement of 15 rows and 27 required checks. All three tools take a page argument, and refuse rather than pick one when a file holds more than one statement.

Install

Fresh machine, clean virtual environment, Python 3.10 or newer (mcp 2.x needs 3.10; developed and run locally on 3.14; the CI matrix covers 3.10 to 3.13):

cd docverify-mcp                   # your local copy; nothing is published to clone yet

python -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate

pip install -r requirements.txt
pytest

Expect 2977 tests: 2976 passed, 1 skipped, in about six minutes. The skip is one malformed-input case that has no inline equivalent, and it reports itself as skipped.

Run plain pytest, not pytest -q: pytest.ini already sets -q, so a second -q composes to -qq and suppresses the pass/fail summary line entirely — you get a wall of dots, no counts, and no way to tell 2100 passed from 3 passed.

requirements.txt pins mcp>=2.0,<3. The 1.x SDK had a different server API and this repo will not import against it.

Reproduce both paths without Claude Desktop

tools/mcp_client_demo.py starts the server as a subprocess and drives it with a real MCP client over real stdio. It exits non-zero if the clean report fails to verify, if the corrupted one is not refused, or if any tool discloses a figure the refusal withheld — so it is a check and not only a demo.

python tools/mcp_client_demo.py             # both paths
python tools/mcp_client_demo.py --happy     # the clean report only
python tools/mcp_client_demo.py --refusal   # the corrupted report only
python tools/mcp_client_demo.py --path other.pdf --page 10

Both bundled reports contain two statements, so the demo names page=10 on every call; a file holding more than one statement is refused rather than guessed at.

The happy path reports verified: true, 25 of 25 checks passed, and 13 line items. The refusal path reports verified: false, line_items absent, and two failed checks: FOOT.disbursements.special_revenue and CROSSFOOT.public_works.

Use it from Claude Desktop

The config file lives at %APPDATA%\Claude\claude_desktop_config.json on Windows and ~/Library/Application Support/Claude/claude_desktop_config.json on macOS. It often does not exist yet — Claude Desktop only writes it once you have used Settings -> Developer -> Edit Config, so create it if it is missing. If it already exists and already has an mcpServers object, merge the "docverify" entry into it; replacing the whole object silently removes every other server you had configured.

Replace ABSOLUTE_PATH_TO_REPO with the full path to your copy.

Windows (backslashes must be doubled inside JSON):

{
  "mcpServers": {
    "docverify": {
      "command": "ABSOLUTE_PATH_TO_REPO\\.venv\\Scripts\\python.exe",
      "args": ["ABSOLUTE_PATH_TO_REPO\\run_server.py"]
    }
  }
}

macOS / Linux:

{
  "mcpServers": {
    "docverify": {
      "command": "ABSOLUTE_PATH_TO_REPO/.venv/bin/python",
      "args": ["ABSOLUTE_PATH_TO_REPO/run_server.py"]
    }
  }
}

Restart Claude Desktop fully (quit it, do not just close the window), then ask it to extract ABSOLUTE_PATH_TO_REPO/data/sample_report.pdf, and then ABSOLUTE_PATH_TO_REPO/data/corrupted_report.pdf.

If it does not connect, the logs are the fastest way to find out why:

platform	log
Windows	%APPDATA%\Claude\logs\mcp-server-docverify.log and mcp.log
macOS	~/Library/Logs/Claude/mcp-server-docverify.log and mcp.log

The usual causes are a command that is not an absolute path to a Python that has the requirements installed, single backslashes in the Windows JSON, and a config file that is not valid JSON after hand-editing.

run_server.py puts the repo root on sys.path itself, so there is no working-directory assumption and the package does not need to be installed. That is checked by tests/test_mcp_stdio.py, which launches it from an unrelated working directory.

What the refusal looks like

data/corrupted_report.pdf is the real report with one figure changed inside the PDF. tools/make_corrupt_pdf.py reads the page's content stream, maps the embedded Type0 font's CIDs back to characters, and overwrites the seven glyphs of 150,717 in the Special Revenue column with 150,171. Same glyph count, and Arial's digits share one advance width, so nothing about the layout moves. The extractor then reads a genuinely wrong number out of a genuinely valid PDF.

Every block in this section is printed verbatim by python tools/mcp_client_demo.py, line for line, and tests/test_readme_claims.py runs the demo and checks that.

verified          : False
line_items key    : ABSENT (refused)
reason            : verification_failed
message           : Refused to return figures: 2 of 25 tie-out check(s) failed: FOOT.disbursements.special_revenue, CROSSFOOT.public_works. The extracted statement is not proven, so no line-item values are returned. The failing arithmetic, if any, is below.
  [x] FOOT.disbursements.special_revenue: 42,423 + 56,082 + 150,171 + 0 = 248,676   vs reported 249,222   delta -546
  [x] CROSSFOOT.public_works: 0 + 150,171 = 150,171   vs reported 150,717   delta -546

and explain_exception("CROSSFOOT.public_works"):

  DISAGREES: computed 150,171 vs reported 150,717, difference -546 (tolerance 1.0)
  computed 150,171 from page(s) [10]
    0 + 150,171
  reported 150,717 on page 10 (Public Works, Combined Total)

Then, in the same session, with no path argument and nothing else changed:

verify_totals()   : 2 check(s) disclosed, 23 withheld
                    This document did not verify, so disclosure is scoped to the failure diagnosis: the arithmetic o...
explain_exception('FOOT.receipts.general') -> disclosed: False
                    explanation key: ABSENT (withheld)
cells reachable   : 7 (the failing checks' operands), 0 beyond the refusal

That last block is the part that took two rounds to get right, and it is the reason the property is stated about the server rather than about extract_statement.

Rebuild the fixture at any time with python tools/make_corrupt_pdf.py --verify.

Testing

pytest                             # the suite
python prove_checks_can_fail.py    # break each guarantee, require red, restore, require green

tests/test_input_envelope.py runs each malformed-input CLASS through each tool — unusable PDFs, unusable paths, and 22 shapes of malformed JSON — and requires a structured refusal that names the offending line item and column, never a bare exception.

Forty-five breaks. That is what prove_checks_can_fail.py performs: it breaks each guarantee in turn, requires the tests to go red, restores, and requires green again. A suite that cannot go red is not evidence, so this runs in CI beside the suite itself.

CI runs the suite and then runs that prover.

Layout

path	what it is
docverify/extract.py	from upstream, with TWO marked divergences: it records ambiguous cells, and its page geometry is derived and self-checking rather than three constants
docverify/anchors.py	new: the five anchor rows and the two section selectors, in ONE place for the precondition, the structured checks and the vendored validator
docverify/validate.py	vendored from upstream: the authoritative tie-out checks (anchor patterns imported, not re-written)
docverify/identity.py	new: ONE canonical identity for a name, and the injectivity rule its check ids must obey
docverify/checks.py	new: the same checks as structured records, plus the relation-coverage gate
docverify/structure.py	new: the structural preconditions, the expected check manifest, label normalization, statement-page detection
docverify/prose.py	new: every verdict sentence, and the one quoting helper document text enters a sentence through
docverify/inputs.py	new: the input boundary — every input becomes a statement or an InputError
docverify/contract.py	new: the disclosure decision, the verdict, the payload shapes
docverify/server.py	new: the MCP server, stdio transport
run_server.py	launcher a client points at
tools/make_corrupt_pdf.py	builds the corrupted fixture by editing the PDF content stream
tools/make_statement_pdf.py	synthesizes born-digital Combined Statement PDFs (reportlab)
tools/mcp_client_demo.py	real MCP client over stdio, drives both paths
prove_checks_can_fail.py	breaks each guarantee, requires red, restores, requires green
tests/test_structure.py	the perturbation family, the manifest, the preconditions
tests/test_prose.py	no payload claims a verification the verdict does not support
tests/test_identity.py	the twin family: same name and refused, or different names and different ids
tests/test_confusables.py	the confusable family: mixed script and shared skeleton, both axes, both input paths
tests/test_ambiguous_cells.py	the rendering family: the printed figure exactly, or a refusal
tests/test_column_shapes.py	the column-shape family: no finding that is not an error
tests/test_manifest_anchor.py	the manifest names every expected check exactly once
tests/test_caller_text.py	the caller's arguments are data; an unknown id answers found: false
tests/test_prose_injection.py	engineered claim strings in every document-supplied field
tests/test_disclosure.py	the server-wide property, over the whole call surface
tests/test_claim_separation.py	the two claims, and the relation-coverage gate
tests/test_input_envelope.py	every malformed-input class, through every tool
tests/test_roundtrip.py	the output round-trips as input; zero checks is never a pass
tests/test_refusal.py	the refusal fires, because the input changed
tests/test_contract_shape.py	the payload shapes, and the guards' teeth
tests/test_parity.py	the structured checks cannot drift from validate.py
tests/test_golden.py	the extraction, pinned figure for figure
tests/test_mcp_stdio.py	it serves, over real stdio
claude-pack/	a Claude Code skill and two slash commands, plus their installer
data/sample_report.pdf	the public AOS report
data/corrupted_report.pdf	the same report with one figure overwritten in the PDF
data/expected_golden.json	the pinned extraction
.github/workflows/ci.yml	the suite and the prover

Claude Code pack

A skill and two slash commands, written for this repo:

python claude-pack/install_pack.py            # into ./.claude here
python claude-pack/install_pack.py --user     # into ~/.claude everywhere
python claude-pack/install_pack.py --list     # dry run

/verify-report <path> and /explain-exception <check_id>, plus a tie-out-review skill whose entire job is the one rule: never state a figure that did not arrive under verified: true. See claude-pack/README.md.

Provenance and license

Extraction and validation are reused, not re-derived. docverify/extract.py and docverify/validate.py are vendored from ethan-haas/financial-statement-extractor at commit 6ffac5b, together with data/sample_report.pdf and data/expected_golden.json. Same author, same MIT license.

Precisely: docverify/validate.py is content-identical to upstream modulo line endings, the provenance header, and one import line (import extract -> from . import extract, for packaging).

docverify/extract.py is not, and it has now diverged twice. Both divergences are marked with banners at the top of the file together with their reproductions.

• _ambiguous_cells was added, listing every (row, column) that received more than one value token, without changing any value the module produced.
• The page geometry was replaced. This is a rewrite, not an addition. Upstream's absolute _LABEL_MAX_X = 295 cutoff, its Column %d fallback and its 50pt header window are gone, replaced by a derived, self-checking column model: the extractor establishes the column boundaries from the page it is reading and verifies them, rather than trusting three constants that happen to fit one layout.

What is still upstream's, unchanged: the token grammar (_NUM, parse_amount, is_value_token), the row grouping, the 25pt column-clustering gap, the statement-page test, the header-metadata regexes, the section rule's receipts/disbursements vocabulary, and the answer on the bundled report — data/expected_golden.json is byte-for-byte what the new extractor produces, which is the strongest statement available here and the one tests/test_golden.py and tests/test_parity.py make. Earlier versions of this README described both files as vendored unchanged; that is no longer true of extract.py, and this paragraph is the correction.

docverify/validate.py has one small divergence of its own: its five anchor patterns and its two section selectors were literals, and copies of the same literals lived in docverify/structure.py and docverify/checks.py. Three copies of one predicate is a defect waiting for the vocabulary to widen, which it did. They now live in docverify/anchors.py and all three modules import them. The behaviour on the bundled report is unchanged and tests/test_parity.py is what says so.

The line-ending caveat is real and worth stating: the blobs committed here are LF, but this repository was authored on Windows with core.autocrlf=true, so a Windows checkout materialises them as CRLF and a byte-for-byte diff against upstream on such a checkout will differ on every line. The two data files are byte-identical to upstream. New code here is the MCP layer, the structured check records, the input boundary, the disclosure contract, the corruption tool and the tests.

The sample report is a public record published by the Ohio black-box test of State. Nothing in this repo is client work.

MIT. See LICENSE.