RAG Chat Assistant

A document Q&A Chat Assistant powered by Retrieval-Augmented Generation (RAG). Uses a hybrid retrieval system (semantic + keyword search) with an MCP (Model Context Protocol) server/client architecture, PII redaction, automated evaluation via RAGAS, and full observability tracing.

---

Architecture

Streamlit Chat UI (Client - .venv)
    ↕ MCP Protocol (Streamable HTTP on localhost:8000)
MCP Server (FastMCP - .mcpvenv)
    ├── Tools: filesystem, doc_loader, chunker, ingest, retriever
    ├── Agents: RAG Agent, Summarizer, PII Redactor, Evaluator
    ├── Storage: ChromaDB (vector) + BM25 (keyword) + Registry
    └── External: Ollama (LLM + Embeddings), Opik (Observability)

Project Structure

L3June26_Assignment/
├── MCP_Stack/                  # MCP Server (runs in .mcpvenv)
│   ├── agents/
│   │   ├── rag_agent.py           # LangGraph RAG agent (retrieve → generate)
│   │   ├── summarizer_agent.py    # Iterative document summarization with caching
│   │   ├── pii_redactor.py        # Regex + optional LLM-based PII detection
│   │   └── evaluator_agent.py     # RAGAS evaluation + ground-truth generator
│   ├── tools/
│   │   ├── doc_loader.py          # Multi-format document loading (PDF/DOCX/TXT/CSV/XLSX/XML/images)
│   │   ├── chunker.py             # Semantic chunking with metadata
│   │   ├── ingest.py              # Ingestion pipeline + document registry
│   │   ├── retriever.py           # Hybrid search (ChromaDB + BM25 + reranking)
│   │   └── filesystem.py          # Sandboxed file browsing
│   ├── mcp_server.py              # FastMCP server entry point
│   ├── config.py                  # Server configuration
│   ├── .env.example               # Server secrets template
│   ├── requirements_mcp.txt       # Server dependencies
│   ├── knowledge_source/          # Drop documents here for ingestion
│   ├── knowledge_base/            # ChromaDB + BM25 index + registry.json (auto-generated)
│   ├── Server_Logs/               # Per-session JSONL logs
│   └── cache/                     # Summarizer cache (by content hash)
├── tests/                     # All tests
│   ├── test_property_*.py         # Property-based tests (Hypothesis)
│   ├── test_unit_*.py             # Unit tests
│   └── test_integration_*.py      # Integration tests
├── Client_Logs/               # Client JSONL logs
├── streamlit_app.py           # Streamlit chat UI (runs in .venv)
├── config.py                  # Client configuration
├── .env.example               # Client secrets template
├── requirements.txt           # Client dependencies
├── test_tools.py              # Manual test stub for tools/agents
└── README.md

---

Prerequisites

Dependency	Purpose
Python 3.12	Runtime (RAGAS has compatibility issues with 3.14)
uv	Package manager (replaces pip)
Ollama	Local/cloud LLM serving
Tesseract OCR (optional)	Primary OCR for images; if unavailable, falls back to gemma4:31b-cloud vision model

---

Setup

1. Pull Required Ollama Models

# Chat model (cloud-hosted, no local GPU needed)
ollama pull gpt-oss:120b-cloud 

# Embedding model
ollama pull nomic-embed-text

# Vision model (OCR fallback — cloud-hosted, no local GPU needed)
ollama pull gemma4:31b-cloud

2. Create Virtual Environments

MCP Server (.mcpvenv):

uv venv .mcpvenv --python 3.12

# Windows
.mcpvenv\Scripts\activate

# Linux/macOS
source .mcpvenv/bin/activate

uv pip install -r MCP_Stack/requirements_mcp.txt

Streamlit Client (.venv):

uv venv .venv --python 3.12

# Windows
.venv\Scripts\activate

# Linux/macOS
source .venv/bin/activate

uv pip install -r requirements.txt

3. Configure Environment Variables

# Copy templates
cp .env.example .env
cp MCP_Stack/.env.example MCP_Stack/.env

Edit each .env file with your actual values:

Client .env:

OLLAMA_BASE_URL=http://localhost:11434
ORCHESTRATOR_MODEL=gpt-oss:120b-cloud
MCP_SERVER_URL=http://localhost:8000/mcp
ENABLE_OPIK_TRACING=false
OPIK_API_KEY=<your-key>
OPIK_WORKSPACE=<your-workspace>
OPIK_PROJECT_NAME=rag-chat-assistant

Server MCP_Stack/.env:

OLLAMA_BASE_URL=http://localhost:11434
DEFAULT_MODEL=gpt-oss:120b-cloud
EMBEDDING_MODEL=nomic-embed-text
VISION_MODEL=gemma4:31b-cloud
CHUNK_SIZE=2000
CHUNK_OVERLAP=200
RETRIEVAL_TOP_K=5
SEMANTIC_WEIGHT=0.7
PII_USE_LLM=false
ENABLE_RAGAS_EVAL=false
MCP_SERVER_PORT=8000

4. Add Documents to Knowledge Source

Place your documents (PDF, DOCX, TXT, CSV, XLSX, XML, or images) into:

MCP_Stack/knowledge_source/

These will be automatically ingested when the MCP server starts.

---

Running the Application

Step 1: Start the MCP Server

Open a terminal and activate the server environment:

# Windows
.mcpvenv\Scripts\activate

# Linux/macOS
source .mcpvenv/bin/activate

# Start the server
python -m MCP_Stack.mcp_server

On startup, the server will:

1. Inject SSL certificates (truststore)
2. Load existing knowledge base from disk
3. Scan knowledge_source/ and ingest any new or modified documents
4. Skip unchanged documents (based on content hash)
5. Register all tools and agents
6. Serve MCP protocol on http://localhost:8000/mcp

Note: Documents added to knowledge_source/ while the server is running will NOT be auto-detected. Restart the server to ingest new files.

Step 2: Start the Streamlit Client

Open a separate terminal and activate the client environment:

# Windows
.venv\Scripts\activate

# Linux/macOS
source .venv/bin/activate

# Start the UI
streamlit run streamlit_app.py

The chat UI will open in your browser (typically at http://localhost:8501).

---

Usage

Asking Questions

Type your question in the chat input. The RAG agent will:

1. Search the knowledge base using hybrid retrieval (semantic + keyword)
2. Generate an answer with citations to source documents
3. Display RAGAS evaluation scores (if enabled)

Document Management

Through the chat interface you can:

• Browse files — list and inspect documents in knowledge_source/
• Ingest manually — force re-ingest of a specific file or all files
• List documents — see all ingested documents with metadata
• Delete documents — remove a document from the knowledge base
• Summarize — get a concise summary of a long document

Ground-Truth Test Data Generation

Generate evaluation test data from your documents:

1. Provide a document name from knowledge_source/
2. The system generates question-answer pairs with context passages
3. Output is saved as JSON for use with RAGAS evaluation (faithfulness, answer relevancy, context precision, context recall)

---

Configuration Reference

Server Configuration (MCP_Stack/config.py)

Parameter	Default	Description
OLLAMA_BASE_URL	http://localhost:11434	Ollama API endpoint
DEFAULT_MODEL	gpt-oss:120b-cloud	Chat model for answer generation
EMBEDDING_MODEL	nomic-embed-text	Embedding model for vector search
VISION_MODEL	gemma4:31b-cloud	Cloud vision model (OCR fallback)
MAX_TOKENS	2048	Max tokens for generated responses
TEMPERATURE	0.7	LLM temperature
CHUNK_SIZE	2000	Characters per chunk (~500 tokens)
CHUNK_OVERLAP	200	Overlap between consecutive chunks
RETRIEVAL_TOP_K	5	Number of chunks to retrieve
SEMANTIC_WEIGHT	0.7	Semantic vs keyword balance (0.7 = 70% semantic)
PII_USE_LLM	false	Enable LLM-based PII detection (slower, catches more)
ENABLE_RAGAS_EVAL	false	Auto-evaluate responses with RAGAS
MCP_SERVER_PORT	8000	Server port

Client Configuration (config.py)

Parameter	Default	Description
OLLAMA_BASE_URL	http://localhost:11434	Ollama API endpoint
ORCHESTRATOR_MODEL	gpt-oss:120b-cloud	Model for client-side orchestration
MCP_SERVER_URL	http://localhost:8000/mcp	MCP server endpoint
ENABLE_OPIK_TRACING	false	Enable Opik observability tracing

---

Running Tests

# Activate the server environment (has all dependencies)
# Windows
.mcpvenv\Scripts\activate

# Linux/macOS
source .mcpvenv/bin/activate

# Run all tests
python -m pytest tests/ -v

# Run only property-based tests
python -m pytest tests/test_property_*.py -v

# Run only unit tests
python -m pytest tests/test_unit_*.py -v

# Run a specific test file
python -m pytest tests/test_unit_chunker.py -v

---

Key Design Decisions

Decision	Choice	Rationale
Protocol	MCP over Streamable HTTP	Standardized tool/agent interface; single endpoint
Agent Framework	LangGraph	Stateful graph workflows with conditional routing
Vector Store	ChromaDB (persistent)	Local file-based; no external service needed
Keyword Search	rank-bm25 (BM25Okapi)	Lightweight in-process; complements semantic search
Embedding	nomic-embed-text via Ollama	Dedicated embedding model; local inference
OCR	Tesseract → gemma4:31b-cloud fallback	Tesseract is fast; cloud vision is available everywhere
Observability	Opik (by Comet)	Native LangChain callback integration
Evaluation	RAGAS	Standard RAG evaluation framework
SSL	truststore	Corporate proxy support via Windows cert store

---

Troubleshooting

Issue	Solution
SSL errors behind corporate proxy	Ensure truststore is installed and imported first in entry points
Ollama connection refused	Verify Ollama is running: ollama list
Empty OCR results	Install Tesseract, or ensure gemma4:31b-cloud is available via ollama pull gemma4:31b-cloud
MCP connection timeout	Check that the server is running on the configured port (default 8000)
Documents not appearing after adding	Restart the MCP server — ingestion only happens at startup
RAGAS scores not showing	Set ENABLE_RAGAS_EVAL=true in MCP_Stack/.env

---

Supported Document Formats

Format	Extensions	Method
PDF	.pdf	pypdf + pdfplumber fallback
Word	.docx	python-docx
Plain Text	.txt	Direct read with encoding detection
CSV	.csv	pandas
Excel	.xlsx	openpyxl via pandas
XML	.xml	xml.etree + lxml fallback
Images	.png, .jpg, .jpeg, .tiff	Tesseract OCR → gemma4:31b-cloud fallback