Tool of Truth

Every tool call, proven.

Verification sentinel for AI agent tool usage. MITM proxy that intercepts all MCP tool calls, detects fabrication, tracks costs, verifies outcomes, and infers user satisfaction.

Features

Feature	Status	Description
MITM Proxy	✅	Transparently intercepts all MCP tool calls
Fabrication Detection	✅	7-signal deep analysis: trace, doc-similarity, timing, placeholders, contradictions
Trust Scoring	✅	Weighted 0-100 with fabrication cap
Cost Tracking	✅	Per-call cost for 8 providers
Token Counting	✅	Input/output tokens per call
Outcome Verification	✅	Compares tool result against user's original prompt
Skill Adherence	✅	Verifies workflow via manifests
User Satisfaction	✅	Infers satisfaction from follow-up messages
Receipts	✅	SHA-256 hashed, JSON + human-readable
Local Storage	✅	JSONL daily files + indexed lookup
Tool Transparency	✅	Shows what downstream tools actually do

Quick Start

1. Install

npm install -g tooloftruth-mcp

2. Connect (Option A: Direct)

// Your agent's MCP config
{
  "tooloftruth": {
    "command": "tooloftruth-mcp"
  }
}

3. Connect (Option B: Proxy — MITM mode)

// Your agent's MCP config — proxy a single downstream server
{
  "firecrawl": { "command": "npx", "args": ["tooloftruth-mcp", "firecrawl"] }
}

Or proxy every local MCP server via ~/.tooloftruth/proxy.json:

// ~/.tooloftruth/proxy.json
{
  "servers": {
    "firecrawl": { "command": "npx", "args": ["firecrawl-mcp"] },
    "github": { "command": "npx", "args": ["@modelcontextprotocol/server-github"] }
  }
}

Skill Manifests (enforced)

Manifests in ~/.tooloftruth/manifests/<skill>.json gate tool calls:

{
  "skill": "echo",
  "version": "1.0.0",
  "requires": {
    "echo": { "tool": "echo", "must_be_called": true, "expected_args": { "message": "" } }
  },
  "outputRules": ["Output must contain \"Echo:\""],
  "max_cost": 1.0
}

Undeclared tools are blocked (BLOCKED_BY_MANIFEST); arg/pattern/cost/output-rule violations are recorded and downgrade the verdict.

4. Use

You: Did you actually use firecrawl?
Agent: [calls tooloftruth_verify]
Agent: VERIFIED — firecrawl was called at 10:30:02, trust score 98/100, cost $0.03

You: How much did I spend on tools today?
Agent: [calls tooloftruth_cost]
Agent: Total: $0.47 across 12 tool calls

You: Was that result what I asked for?
Agent: [calls tooloftruth_outcome]
Agent: ALIGNED — tool result matches your request

You: That's wrong, try again
Agent: [calls tooloftruth_satisfaction]
Agent: User appears dissatisfied — result may be wrong

CLI

# Intercept and monitor non-MCP commands
tooloftruth -- gh api repos/user/repo

# One-shot status report (daemon health, receipts, cost, top tools)
tooloftruth status

Daemon

A launchd daemon watches .tooloftruth/receipts/, aggregates daily stats, and alerts on fabrications.

# Install the daemon (macOS)
launchctl load ~/Library/LaunchAgents/com.tooloftruth.daemon.plist

Stats are written to .tooloftruth/stats/YYYY-MM-DD.json.

Truth Scan

tooloftruth_truth_scan does real web verification of claims: Bing search (curl) → crawl4ai fetch → evidence extraction → verdict.

Sensitive-Data Detection

The daemon scans every new message and tool call for leaks, writing alerts with exact proof:

• Secrets: 24 gitleaks-style rules + entropy — AWS keys, GitHub/OpenAI/Google/Anthropic tokens, JWTs, private keys, DB connection strings, generic API keys, passwords
• PII: email, phone, SSN, credit card (Luhn-validated), IP, IBAN
• Prompt injection: ignore-previous-instructions, system-prompt reveal, jailbreak (DAN), XML tag injection
• Dangerous commands: rm -rf /, curl | sh, dd to disk, base64-decode-to-shell, etc.

Alerts land in ~/.tooloftruth/alerts/ with redacted match, source, confidence, and surrounding context.

Deep scans: tooloftruth_gitleaks <repo> runs gitleaks (200+ rules) over a repo's git history with exact file:line proof.

Behavior Ledger

The daemon tracks per-session model, message/tool-call counts, error rate, token usage, and cost → ~/.tooloftruth/ledger/. Query via tooloftruth_ledger.

MCP Tools

Tool	Description
tooloftruth_verify	Verify a specific tool was used
tooloftruth_check	Pre-flight: is tool installed/configured?
tooloftruth_receipt	Generate/view verification receipt
tooloftruth_cost	Cost breakdown by tool
tooloftruth_history	Search historical receipts
tooloftruth_truth	Full session truth report
tooloftruth_satisfaction	Infer user satisfaction from message
tooloftruth_outcome	Verify result matches user prompt
tooloftruth_audit	Cross-reference agent claims against actual tool calls
tooloftruth_conversation	View conversation log
tooloftruth_truth_scan	Web research + fact-check a claim
tooloftruth_alerts	List sensitive-data alerts (PII/secrets/injection/commands)
tooloftruth_gitleaks	Deep-scan a git repo for secrets
tooloftruth_ledger	Per-session behavior (model, errors, tokens, cost)

How the MITM Proxy Works

Agent ←→ Tool of Truth ←→ MCP Server (downstream)
              ↓
         Records everything
         Runs deep fabrication checks
         Verifies outcome alignment
         Infers satisfaction
         Generates receipts
         Returns result unchanged

Storage

All data stays in your machine's home directory:

~/.tooloftruth/
├── config.json          ← settings
├── index.json           ← fast lookup index
├── proxy.json           ← downstream server config
├── receipts/
│   ├── 2026-08-17.jsonl ← today's calls
│   └── ...
├── conversations/
│   └── 2026-08-17.jsonl ← claim/action conversations
└── stats/               ← daemon aggregations
    ├── 2026-08-17.json
    └── ...

## Skill Manifests

Skills can attach verification manifests:

```json
{
  "skill": "last-30-days",
  "version": "2.1.0",
  "requires": {
    "github_mcp": {
      "tool": "get_repo_stats",
      "mustBeCalled": true
    }
  }
}

Development

git clone https://github.com/adigoel07/tooloftruth.git
cd tooloftruth
pnpm install
pnpm build
npx vitest run

License

MIT — use it, fork it, build on it.

Author

Built by adigoel07.