Obsidian MCP Wrapper

Obsidian MCP Wrapper is a session-stateless MCP retrieval system for an agency Obsidian vault. It gives Claude Desktop users controlled access to indexed Markdown knowledge without giving the model direct, unbounded filesystem access or unsupervised write privileges.

The runtime has two layers:

• pf-mcp: a lightweight STDIO MCP shim that runs on each Claude Desktop workstation.
• pf-index: a shared LAN/VPN retrieval service that owns the vault index, signed handles, bearer-token authentication, rate limits, write locking, and the audit log.

The design goal is progressive disclosure: search returns compact snippets and signed section handles first, then Claude expands only the context it needs.

Highlights

• Standard-library core runtime for the default HTTP server path.
• Optional FastAPI/uvicorn ASGI boundary.
• SQLite FTS5 lexical search over immutable vault index epochs.
• HMAC-signed section handles that resolve against retained snapshots.
• Manifest-only filesystem access for indexed Markdown paths.
• Human-gated append writes into client, provider, carrier, and operational notes.
• Autonomous AI notes routed to a rotated capture inbox by default.
• File locking and SQLite audit records for write operations.
• Sanitized error envelopes that avoid leaking paths, tokens, URLs, or tracebacks into Claude.
• Quick-start, deployment, teardown, and nightly-refresh scripts.

Runtime Topology

Claude Desktop
  -> STDIO MCP JSON-RPC
  -> pf-mcp workstation shim
  -> HTTP POST /tools/<tool_name>
  -> pf-index LAN/VPN service
  -> Obsidian_Vault Markdown snapshots and .pf_index artifacts

pf-index consumes Markdown and derived index artifacts. It does not need the source DuckDB database at runtime.

Repository Layout

.
|-- Docs/
|   |-- ARCHITECTURE.md
|   |-- DEPLOYMENT.md
|   |-- QUICKSTART.md
|   `-- MCP_WRAPPER_MASTER_PROMPT.md
|-- config/
|   |-- claude_desktop_config.example.json
|   |-- deployment_manifest.example.json
|   |-- pf-index.example.toml
|   `-- pf-mcp.env.example
|-- pf_index/
|   |-- api/                 HTTP boundaries
|   |-- core/                indexing, retrieval, crypto, paths, writes
|   `-- config.py            TOML and environment configuration
|-- pf_mcp/
|   `-- main.py              STDIO MCP shim
|-- scripts/
|   |-- build_index.py
|   |-- run_pf_index.py
|   |-- quickstart.sh
|   |-- quickstart_teardown.sh
|   |-- nightly_refresh.sh
|   |-- deploy_interactive.sh
|   `-- deploy_teardown.sh
|-- tests/
|-- pyproject.toml
`-- requirements.lock

MCP Tools

The shim exposes these tools to Claude:

Tool	Purpose
vault_search	Search indexed vault sections and return snippets with signed handles.
strategy_search	Search strategy, advisory, sales, renewal, and operating-system material.
find_note	Find likely note paths by query.
get_note_summary	Return a bounded summary/outline for one note path.
get_entity_context	Entity-oriented search wrapper.
search_tables	Search indexed table-like content.
read_section	Resolve one signed handle into a larger section excerpt.
expand_context	Resolve several signed handles within a token budget.
get_full_profile	Retrieve full materialized context for one known client or provider.
advisory_context	Combine profile evidence with strategy context for planning questions.
vault_stats	Return index and vault statistics.
append_human_note	Append a factual note through the gated write pipeline.
remove_note	Remove a specific generated note entry by note id.

Security Model

Important boundaries:

• The MCP shim validates argument shapes and hard limits before forwarding to the LAN service.
• The LAN service requires Authorization: Bearer <token>.
• Tokens map to user ids for audit and idempotency.
• Runtime reads are limited to paths listed in the active index manifest.
• Path validation rejects absolute paths, traversal, null bytes, URL schemes, Windows drive paths, and backslash paths.
• Section handles contain path, line range, content hash, file hash, vault id, and index epoch, then are signed with HMAC-SHA256.
• Old handles resolve against retained immutable snapshots, not changed live vault files.
• Autonomous writes are redirected to the configured capture inbox unless the human explicitly directs a write to a specific note.
• Write operations are append-only/gated, locked, audited, and bounded by configured note length limits.

Requirements

• Python 3.11 or newer
• Markdown Obsidian vault directory
• HMAC secret for handle signing
• One or more bearer tokens for users

The default pf_index.api.server HTTP boundary uses only the Python standard library. Optional FastAPI dependencies are listed in requirements.lock:

python -m pip install -r requirements.lock

Quick Start

For a complete local smoke test, use the maintained guide:

sed -n '1,240p' Docs/QUICKSTART.md

The short version:

export PF_APP_DIR="$PWD"
export PF_DEMO_ROOT=/tmp/pf-mcp-demo
export PF_VAULT_ROOT="$PF_DEMO_ROOT/Obsidian_Vault"
export PF_INDEX_ROOT="$PF_DEMO_ROOT/.pf_index"
export PF_AUDIT_DB="$PF_INDEX_ROOT/write_audit.sqlite"
export PF_INDEX_HMAC_SECRET="local-demo-secret-change-me"
export PF_INDEX_TOKEN="local-demo-token"
export PF_INDEX_CONFIG="$PF_DEMO_ROOT/pf-index.toml"

Create a small test vault and config as shown in Docs/QUICKSTART.md, then build an index:

PYTHONPATH="$PF_APP_DIR" python -m scripts.build_index \
  --config "$PF_INDEX_CONFIG" \
  --epoch 2026-06-08T120000Z

Run the stdlib HTTP server:

PYTHONPATH="$PF_APP_DIR" \
PF_INDEX_HMAC_SECRET="$PF_INDEX_HMAC_SECRET" \
python -m pf_index.api.server \
  --host 127.0.0.1 \
  --port 8765 \
  --config "$PF_INDEX_CONFIG"

Search through the API:

curl -s \
  -H "Authorization: Bearer $PF_INDEX_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"query":"Coverage","max_results":3,"max_tokens":1000}' \
  http://127.0.0.1:8765/tools/vault_search \
  | python -m json.tool

List MCP tools through the shim:

printf '%s\n' '{"jsonrpc":"2.0","id":1,"method":"tools/list","params":{}}' \
  | PYTHONPATH="$PF_APP_DIR" \
    PF_INDEX_URL="http://127.0.0.1:8765" \
    PF_INDEX_TOKEN="$PF_INDEX_TOKEN" \
    python -m pf_mcp.main \
  | python -m json.tool

Or run the automated path:

bash scripts/quickstart.sh

Tear down demo artifacts:

bash scripts/quickstart_teardown.sh

Configuration

Server-side config lives in TOML. Start from:

cp config/pf-index.example.toml pf-index.toml

Key sections:

• [paths]: vault_root, index_root, and audit_db
• [limits]: result, token, note length, lock timeout, and concurrency limits
• [security]: vault_id and HMAC secret source
• [tokens]: bearer token to user id map
• [inbox]: rotated inbox path for autonomous captures
• [retention]: retained immutable index epochs

Workstation shim config is environment-based. Start from:

cp config/pf-mcp.env.example pf-mcp.env

Claude Desktop config example:

cat config/claude_desktop_config.example.json

Indexing

Build an immutable index epoch:

PYTHONPATH="$PWD" python -m scripts.build_index --config /path/to/pf-index.toml

Each epoch contains:

.pf_index/builds/<epoch>/
|-- manifest.json
|-- lexical.sqlite
`-- content/<sha256-of-relative-path>.md

The active epoch is selected through .pf_index/ACTIVE. Rolling back is a controlled edit of that pointer to a retained epoch.

Running The Service

Default stdlib server:

PYTHONPATH="$PWD" python -m pf_index.api.server \
  --host 127.0.0.1 \
  --port 8765 \
  --config /path/to/pf-index.toml

Optional FastAPI boundary:

python -m pip install -r requirements.lock
PYTHONPATH="$PWD" uvicorn pf_index.api.fastapi_app:create_app --factory \
  --host 127.0.0.1 \
  --port 8765

Testing

Run acceptance and structure tests:

PYTHONDONTWRITEBYTECODE=1 python -m unittest discover -s tests -v

The tests focus on safety contracts such as path validation, signed handles, indexing behavior, write routing, and repository structure.

Deployment

Use Docs/DEPLOYMENT.md for the production runbook. It covers:

• Dedicated service user and filesystem permissions
• Code and vault placement
• HMAC secret generation
• User token generation
• Index build and startup checks
• systemd service setup
• LAN/VPN binding and firewall posture
• Runtime egress denial
• Token rotation
• Audit recovery
• Rollback

scripts/deploy_interactive.sh and scripts/deploy_teardown.sh support guided deployment and cleanup flows.

Nightly Refresh

scripts/nightly_refresh.sh supports a scheduled flow:

sync or refresh source data
  -> optionally rebuild Markdown vault from DuckDB
  -> build a new immutable index epoch
  -> atomically update ACTIVE

If DuckDB-to-Markdown generation happens elsewhere, omit PF_SOURCE_DATABASE and rebuild the index from the current vault.

Data Boundary

Do not commit or deploy local data artifacts:

• agency_core.duckdb
• Generated Obsidian_Vault/
• .pf_index/ builds
• .obsidian/
• .vault_salt
• Real .env files
• Audit databases
• Exported CSVs, spreadsheets, or reports

The repository is meant to contain application code, tests, docs, and placeholder configuration only.

Further Reading

• Docs/ARCHITECTURE.md for the full design and safety model
• Docs/QUICKSTART.md for local smoke testing
• Docs/DEPLOYMENT.md for production operations
• Docs/MCP_WRAPPER_MASTER_PROMPT.md for operator-facing MCP usage guidance