🔒 Privacy-Preserving Local RAG with MCP Integration

A production-grade, fully local Retrieval-Augmented Generation (RAG) system integrated with the Model Context Protocol (MCP). Zero data ever leaves your machine.

Architecture

User CLI  →  Agent Orchestrator  →  MCP Client (stdio)
                                          ↓
                               ┌──────────────────────┐
                               │     MCP Server        │
                               │  ┌────────────────┐  │
                               │  │  vector_search  │  │
                               │  │  local_db_query │  │
                               │  └────────────────┘  │
                               └──────────────────────┘
                                    ↓           ↓
                               Qdrant DB    SQLite DB
                               (Docker)   (employees)

Tech Stack

Component	Technology
LLM Inference	Ollama (Llama 3)
Embeddings	Ollama (nomic-embed-text, 768d)
Vector Store	Qdrant (Docker)
Re-ranking	Cross-Encoder (ms-marco-MiniLM-L-6-v2)
MCP Server	FastMCP (Python SDK v1.x)
Chunking	SemanticChunker (LangChain Experimental)
Config	Pydantic Settings
Logging	structlog (JSON structured)

Prerequisites

• Python 3.11+
• Docker & Docker Compose
• Ollama installed and running
• At least 8GB RAM (16GB recommended)

Quick Start

1. Clone & Configure

git clone <repo-url>
cd local-rag-mcp
cp .env.example .env

2. Start Qdrant

docker compose up -d
# Verify: open http://localhost:6333/dashboard

3. Pull Ollama Models

# On Linux/macOS:
bash scripts/pull_models.sh

# On Windows (PowerShell):
ollama pull llama3
ollama pull nomic-embed-text

4. Create Python Environment

python -m venv .venv

# Linux/macOS:
source .venv/bin/activate

# Windows:
.venv\Scripts\activate

pip install -r requirements.txt

5. Seed the SQLite Database

python scripts/seed_database.py

6. Ingest Documents

python main.py
# Inside the REPL:
> /ingest data/documents

7. Run the Agent

python main.py
> What does our documentation say about authentication?
> List all employees in the Engineering department
> How many engineers earn above the department average?

Project Structure

local-rag-mcp/
├── docker-compose.yml       # Qdrant container
├── .env.example             # Config template
├── requirements.txt         # Pinned deps
├── pyproject.toml           # Project metadata & tool config
│
├── config/
│   └── settings.py          # Centralized Pydantic config
│
├── data/
│   ├── documents/           # Drop PDFs/Markdown here
│   └── sqlite/
│       └── employees.db     # Auto-seeded SQLite DB
│
├── scripts/
│   ├── seed_database.py     # Seeds employee DB
│   └── pull_models.sh       # Ollama model helper
│
├── src/
│   ├── ingestion/           # PDF/MD loader, semantic chunker, embedder
│   ├── mcp_server/          # FastMCP server + vector_search + local_db_query tools
│   ├── agent/               # Agent loop + HyDE
│   └── utils/               # Structured logging
│
├── tests/                   # Pytest test suite
└── main.py                  # CLI REPL entry point

CLI Commands

Command	Description
/ingest <path>	Ingest all PDFs/Markdown from directory
/hyde on|off	Toggle HyDE query enhancement
/help	Show available commands
/quit	Exit the application
Any other text	Ask the agent a question

Configuration

All settings are controlled via .env. Key variables:

Variable	Default	Description
OLLAMA_LLM_MODEL	llama3	LLM for generation & tool calling
OLLAMA_EMBEDDING_MODEL	nomic-embed-text	Embedding model
QDRANT_COLLECTION_NAME	rag_documents	Qdrant collection name
HYDE_ENABLED	true	Enable HyDE query enhancement
RERANKER_TOP_K	5	Number of final results after re-ranking

Running Tests

pytest tests/ -v

License

MIT