RAG System

Local Retrieval-Augmented Generation system for PDF documentation. Provides offline document ingestion with hybrid vector search and an MCP server for LLM agents to query indexed documentation.

Architecture

                 Offline / manual
PDFs and docs -> Ingestion CLI -> Local Qdrant (Docker)
                                      |
                 During questions     |
LLM agent -> MCP retrieval tools -----+
          -> grounded answer with citations

Two executables share a common codebase:

• rag - Ingestion CLI for parsing, chunking, embedding, and storing PDFs
• rag-mcp - Read-only MCP stdio server exposing search and retrieval tools

Prerequisites

• Python 3.11+
• uv package manager
• Docker (for Qdrant)

Setup

1. Start Qdrant

cd rag-system
docker compose up -d

Qdrant will be available at http://localhost:6333.

2. Install Python dependencies

uv sync

This creates a virtual environment and installs all dependencies including local embedding models.

3. Copy environment configuration

cp .env.example .env

Edit .env if you need to change Qdrant URL, collection name, or model settings.

Ingesting Documents

Place PDF files in the documents/ directory (or reference them by path).

# Ingest a single file
uv run rag ingest ./documents/manual.pdf

# Ingest with a custom document ID
uv run rag ingest ./documents/manual.pdf --id product-manual

# Force re-ingestion (replaces existing version)
uv run rag ingest ./documents/manual.pdf --force

# Ingest all PDFs in a directory
uv run rag ingest-directory ./documents

# List indexed documents
uv run rag list

# Inspect a document's metadata and sample chunks
uv run rag inspect product-manual

# Delete a document
uv run rag delete product-manual

Embedding models (BAAI/bge-small-en-v1.5 for dense, Qdrant/bm25 for sparse) are downloaded on first use and cached locally.

MCP Server Configuration

Add the RAG MCP server to your agent's MCP configuration:

opencode (opencode.json or ~/.config/opencode/opencode.json)

{
  "mcp": {
    "servers": {
      "rag-documentation": {
        "type": "local",
        "command": ["uv", "run", "--directory", "/absolute/path/to/rag-system", "rag-mcp"],
        "enabled": true
      }
    }
  }
}

Claude Desktop (claude_desktop_config.json)

{
  "mcpServers": {
    "rag-documentation": {
      "command": "uv",
      "args": ["run", "--directory", "/absolute/path/to/rag-system", "rag-mcp"]
    }
  }
}

MCP Tools

The server exposes four read-only tools:

Tool	Description
search_documentation	Hybrid semantic + keyword search with page citations
get_document_context	Retrieve surrounding chunks for more context
list_documents	List all indexed documents
get_document_metadata	Get metadata for a specific document

Configuration

All settings are read from environment variables (or .env file):

Variable	Default	Description
QDRANT_URL	http://localhost:6333	Qdrant REST API URL
QDRANT_COLLECTION	documentation	Qdrant collection name
DENSE_EMBEDDING_MODEL	BAAI/bge-small-en-v1.5	Dense embedding model
SPARSE_EMBEDDING_MODEL	Qdrant/bm25	Sparse/BM25 embedding model
CHUNK_TARGET_TOKENS	700	Target tokens per chunk
CHUNK_OVERLAP_TOKENS	100	Token overlap between chunks
DEFAULT_SEARCH_LIMIT	8	Default results per search
MAX_SEARCH_LIMIT	20	Maximum results per search

Development

# Run tests (unit tests run without Qdrant)
uv run pytest tests/unit

# Run all tests (requires Qdrant running)
docker compose up -d
uv run pytest

# Lint
uv run ruff check src tests

# Type check
uv run mypy src

How Retrieval Works

1. Hybrid search: Dense vector search (semantic similarity) and sparse BM25 search (keyword matching) run in parallel against Qdrant.
2. Reciprocal Rank Fusion (RRF): Qdrant fuses dense and sparse results using RRF.
3. Citations: Every result includes the document title, section path, and page range.
4. Context expansion: The get_document_context tool retrieves neighboring chunks when a search result needs more surrounding text.

All processing is local. No document content is sent to external services.