Corpus — shared docs, answered by agents

One Postgres-backed knowledge base with four doors into it: a web app (streaming cited chat, hybrid search, analytics), an MCP server for coding agents (API key or OAuth 2.1 sign-in), a voice agent on a phone number, and Postgres + pgvector underneath — same corpus, same RBAC, every door.

▶ Watch the demo (1:48) — login → search → streaming chat with citations → chunk seams → MCP + OAuth consent → live voice-agent call → admin.

Agent A ──┐                          ┌── chunks: vector(1536) + tsvector
Agent B ──┼── MCP :8765 ──┐          │
Humans ───┼── Web :8000 ──┼── core ──┼── documents (owner, visibility, tags)
Callers ──┘  (SPA + SSE)  │          └── users / oauth / chat history
              voice tool ─┘

The RAG pipeline

Stage	What	Why
Chunking	Markdown header-aware with section paths; code/JSON/YAML at line boundaries; sentence windows elsewhere. Per-document strategy switchable (chonkie: recursive/token/sentence/semantic)	Chunks are coherent units, not 512-token slices
Contextual embeddings	Embeds "{title} > {section}\n\n{chunk}", stores the raw chunk	Cheap contextual retrieval; real recall lift
Hybrid retrieval	pgvector HNSW cosine + Postgres full-text, fused with RRF (k=60); a similarity floor makes "found nothing" possible	Keyword leg catches exact identifiers; the floor feeds the documentation-gaps panel (unanswered questions = your writing backlog)
Answering	Everything is a deepagents agent. fast: one agent, seconds. deep: planner + parallel researcher subagents + adversarial verifier, ~10–40× cost. auto routes; deep failures fall back to fast	An agent that sees an empty result can rephrase; a fixed pipeline can't
Citations	Every chunk has a stable 8-char cite_key derived from (document, position) — re-chunking preserves it. UI renders numbered chips deep-linking to /docs/{id}#chunk-{ord}	Citations in old chat history still resolve after re-indexing

Chat streams over SSE (mode → step/token* → done); agent tool-calls appear as live pipeline steps and a collapsible trace. Design notes: docs/DEEP_AGENT_RAG.md.

Access model

Everything resolves to a Principal (user + role + connection path):

• Web session — Argon2id passwords, throttled logins, signed-cookie sessions.
• MCP API key (sdr_…) — bound to a user; agents are users.
• OAuth 2.1 — for MCP clients with nowhere to paste a key. The web app is the authorization server: PKCE mandatory, Dynamic Client Registration (RFC 7591), single-use 60s codes (replay revokes every derived token), rotating refresh tokens, all tokens stored as sha256. The token is bound to whoever approved the consent screen — no service account, no default admin. Users disconnect clients from their account page. See core/oauth.py.
• Voice — Sarvam Samvaad calls POST /api/agent/ask; answers come back TTS-friendly with citation markup stripped.

Roles viewer < editor < admin; documents are org or private — enforced in every SQL query, so no door can leak across the boundary.

Web app

React 18 + TS + Vite + Tailwind v4 (shadcn-style primitives, Inter/JetBrains Mono, dark + light). Django 5 (ASGI) serves the built SPA and the JSON API; the ORM runs over managed = False models — schema.sql owns the DDL.

Dashboard (stats, gaps, charts) · Chat (streaming, mode pills, trace, citation chips) · Search (visible vec/kw/rrf ranks, per-doc re-chunking, admin-frozen retrieval config shared by chat + MCP + ingest) · Documents · Doc viewer (TOC, related docs, backlinks, chunk-seams overlay) · Admin (roles, keys, audit log) · Connect + voice pages.

Quick start

bash run_local.sh          # zero-config: offline embeddings, no keys needed

Full setup: cp .env.example .env (set GOOGLE_API_KEY), then

docker compose up -d                                   # Postgres + pgvector
pip install -r requirements.txt
(cd webapp/ui && npm ci && npm run build)
uvicorn server.asgi:application --port 8000            # web
python -m mcp_server.server                            # MCP

First account = admin. Mint agent keys in People, wire them per docs/mcp-client-setup.md. Providers are swappable per layer (EMBEDDINGS_PROVIDER, ANSWER_PROVIDER); keep EMBEDDINGS_DIM in sync with vector(1536) in schema.sql if you change models.

Deploy

• Free, no server: render.yaml click-deploys web + MCP to Render; database on Neon (pgvector is why this works on managed free tiers). docs/DEPLOYMENT.md Option A, including corpus migration via scripts/migrate_corpus.py.
• One VM: bash deploy.sh — Caddy (auto-TLS) → web + MCP + Postgres in containers, one Dockerfile for both services.

MCP tools

search_docs · ask_docs · deep_research · upload_document · list_documents · read_document · delete_document · whoami — the tool list an agent sees is filtered by its role, and enforced again on call.

Layout

core/         config · security (Principal/RBAC/keys) · oauth (OAuth 2.1 AS)
              citations · analytics
core/rag/     chunking · embeddings · ingest · retrieve (hybrid+RRF)
              fast · deep_agent · agent_tools · answer (routing + SSE)
mcp_server/   FastMCP HTTP server (the agents' door)
server/       Django ASGI: JSON API + SSE, serves the SPA
webapp/ui/    React SPA
schema.sql    DDL source of truth (auto-applied on first boot)
migrations/   for existing installs (013 = move to pgvector)
docs/         deployment · MCP client setup · deep-agent design · media