🚀 Modular RAG System

A Modular Retrieval-Augmented Generation (RAG) System with Adaptive Retrieval, Intent-Aware Query Processing, Semantic Query Rewriting, and MCP Server Integration

<p align="center">

</p>

---

📖 Overview

Modern enterprise organizations store thousands of pages of policies, reports, technical documents, and internal knowledge bases. Traditional keyword search often fails when users ask questions using natural language, paraphrased wording, or indirect references.

This project implements a Modular Retrieval-Augmented Generation (RAG) System capable of answering natural language questions over enterprise documents while providing grounded, citation-backed responses.

Unlike conventional RAG systems that treat every query identically, this system introduces intent-aware routing, allowing different categories of questions to follow specialized processing pipelines. The architecture further improves retrieval quality through an Adaptive Retrieval Loop, which selectively invokes a local Phi-3 query rewriter only when retrieval confidence is low.

The system follows a layered design emphasizing retrieval quality, grounding, modularity, maintainability, and explainability.

---

🎯 Project Objectives

The primary objectives of this project are:

• Build an MCP-compatible document question answering backend.
• Support multiple enterprise documents simultaneously.
• Generate grounded responses with citations.
• Reduce hallucinations through evidence validation.
• Improve semantic retrieval for paraphrased questions.
• Keep the architecture modular for future scalability.
• Maintain compatibility with local LLMs and cloud LLMs.
• Optimize retrieval quality while minimizing unnecessary latency.

---

✨ Key Features

Intelligent Intent Routing

Instead of treating every user query identically, the system automatically classifies incoming questions into specialized pipelines.

Supported query categories include:

• Factual Questions
• Numeric Questions
• Boolean Questions
• Procedure Questions
• Summary Requests
• Comparison Questions
• Multi-document Queries
• Out-of-Scope Detection

---

Adaptive Retrieval

Retrieval is no longer a single-pass process.

When confidence is high:

User Query
      │
      ▼
 Retrieval
      │
      ▼
 Answer

When confidence is low:

User Query
      │
      ▼
 Initial Retrieval
      │
      ▼
Confidence Evaluation
      │
      ▼
Query Rewriter (Phi-3)
      │
      ▼
Second Retrieval
      │
      ▼
Quality Comparison
      │
      ▼
Best Retrieval Selected

This selective retrieval strategy improves semantic matching while avoiding unnecessary model execution.

---

Query Rewriting

Instead of maintaining thousands of manually curated synonym mappings, the system leverages a local Phi-3 model to rewrite ambiguous queries into document-friendly terminology.

Example:

Original Query

What's the daily food budget?

Rewritten Query

What is the daily meal reimbursement limit during business travel?

The rewritten query is only used if retrieval quality objectively improves.

---

Rewrite Validation

To prevent semantic drift, every rewritten query undergoes strict validation.

Validation rules include:

• Number preservation
• Currency preservation
• Percentage preservation
• Date preservation
• Entity preservation
• Acronym preservation
• Logical operator preservation
• Polarity preservation

Example:

Original

Can employees NOT claim cash gifts?

Rejected Rewrite

Can employees claim cash gifts?

The validator immediately rejects such rewrites.

---

Rewrite Cache

Repeated rewrites are avoided using an LRU cache.

Cache Key

Hash(
    Normalized Query
    +
    Document Collection Hash
)

Benefits

• Reduced latency
• Reduced Phi-3 execution
• Deterministic behaviour
• Automatic invalidation on document changes

---

🏗 System Architecture

                        ┌────────────────────┐
                        │       User         │
                        └─────────┬──────────┘
                                  │
                                  ▼
                     ┌────────────────────────┐
                     │     MCP Server         │
                     └─────────┬──────────────┘
                               │
                               ▼
                  ┌─────────────────────────────┐
                  │ Intent Detection & Routing  │
                  └──────┬───────────┬──────────┘
                         │           │
                         ▼           ▼
              Standard Pipeline   Summary Pipeline
                         │
                         ▼
              Comparison / Numeric /
             Boolean / Procedure etc.
                         │
                         ▼
              Adaptive Retrieval Layer
                         │
          ┌──────────────┼──────────────┐
          │              │              │
          ▼              ▼              ▼
    Query Rewriter   Validator      Rewrite Cache
          │              │              │
          └──────────────┼──────────────┘
                         ▼
                 Final Retrieval
                         │
                         ▼
              Evidence Validation
                         │
                         ▼
             Gemini 2.5 Flash Generator
                         │
                         ▼
              JSON Response + Citations

---

🧠 Layered RAG Pipeline

Document Upload
       │
       ▼
Document Parsing
       │
       ▼
Chunk Generation
       │
       ▼
Embedding Generation
       │
       ▼
ChromaDB Storage
       │
──────────────────────────────────────────
                Runtime
──────────────────────────────────────────
       │
User Query
       │
       ▼
Intent Detection
       │
       ▼
Vector Retrieval
       │
       ▼
Adaptive Retrieval (Optional)
       │
       ▼
Evidence Validation
       │
       ▼
LLM Generation
       │
       ▼
Grounded JSON Response

---

🧩 Core Components

Component	Responsibility
MCP Server	Exposes the QA tool to MCP clients
Intent Router	Routes queries to specialized pipelines
Retriever	Retrieves relevant chunks from ChromaDB
Query Rewriter	Improves low-confidence semantic retrieval
Rewrite Validator	Prevents semantic drift
Rewrite Cache	Avoids repeated rewrites
Evidence Validator	Ensures answer grounding
Gemini Generator	Produces grounded final responses
Phi-3	Local semantic query optimization

---

📂 Project Structure

modular-rag-system/

│
├── data/
│   ├── company_policy.txt
│   ├── conduct_policy.txt
│   ├── finance_policy.txt
│   ├── it_policy.pdf
│   └── remote_work_agreement.pdf
│
├── src/
│   ├── answer_question.py
│   ├── mcp_server.py
│   ├── query_rewriter.py
│   ├── retrieve.py
│   ├── embed_document.py
│   ├── chunk_document.py
│   ├── read_document.py
│   ├── evidence_engine.py
│   ├── layer2_validator.py
│   ├── store_embeddings.py
│   ├── verify_storage.py
│   ├── config.py
│   ├── test_acceptance_suite.py
│   ├── test_query_rewriter.py
│   └── ...
│
├── requirements.txt
├── README.md
└── .gitignore

---

⚙ Technology Stack

Category	Technology
Language	Python
Vector Database	ChromaDB
Embedding Model	all-MiniLM-L6-v2
Primary LLM	Gemini 2.5 Flash
Local Model	Phi-3 (Ollama)
Protocol	Model Context Protocol (MCP)
Retrieval	Dense Vector Search
Validation	Rule-Based Evidence Validation
Environment	VS Code + Python Virtual Environment

---

🚀 Installation

Clone the repository

git clone https://github.com/naffss-eng/modular-rag-system.git

cd modular-rag-system

Create a virtual environment

python -m venv venv

Activate the environment

Windows

venv\Scripts\activate

Linux

source venv/bin/activate

Install dependencies

pip install -r requirements.txt

Create a .env file

GOOGLE_API_KEY=YOUR_API_KEY

▶ Running the Project

Step 1 — Ingest Documents

Load and preprocess the documents.

python src/read_document.py

---

Step 2 — Chunk Documents

Generate semantic chunks.

python src/chunk_document.py

---

Step 3 — Generate Embeddings

python src/embed_document.py

---

Step 4 — Store Embeddings

python src/store_embeddings.py

---

Step 5 — Verify Vector Database

python src/verify_storage.py

---

Step 6 — Start the MCP Server

python src/mcp_server.py

---

Step 7 — Connect Using Antigravity CLI

antigravity connect localhost:8000

Once connected, users can directly ask questions over the uploaded document collection.

---

💬 Example Queries

Factual

What are the company's core collaboration hours?

---

Numeric

What is the daily meal reimbursement limit?

---

Boolean

Are cash gifts allowed?

---

Procedure

What documents are required to submit a travel reimbursement claim?

---

Summary

Give me an overview of the IT Policy.

---

Comparison

Compare the Finance Policy with the Company Policy.

---

Multi-document

What are the remote work responsibilities mentioned across all documents?

---

Out-of-Scope

How do I bake a chocolate cake?

Expected behaviour:

Grounding Status : REJECTED

Answer:
I could not find this information in the uploaded documents.

---

📊 Supported Query Types

Query Type	Supported
Factual	✅
Numeric	✅
Boolean	✅
Procedure	✅
Summary	✅
Comparison	✅
Multi-document	✅
Out-of-Scope Detection	✅

---

🔄 Intent-Aware Processing

Unlike traditional RAG systems that process every query through a single retrieval pipeline, this project dynamically routes each request based on its semantic intent.

                 User Query
                      │
                      ▼
             Intent Classification
                      │
      ┌───────────────┼────────────────┐
      │               │                │
      ▼               ▼                ▼
 Summary         Comparison       Standard QA
      │               │                │
      ▼               ▼                ▼
 Dedicated      Balanced Dual     Adaptive
 Pipeline        Retrieval         Retrieval

This modular routing significantly improves maintainability while allowing future pipelines to be integrated independently.

---

🔍 Adaptive Retrieval Loop

One of the major architectural enhancements introduced in Backend v2.0 is the Adaptive Retrieval Loop.

Instead of rewriting every query, the system first evaluates retrieval confidence.

                    User Query
                         │
                         ▼
                 Initial Retrieval
                         │
                         ▼
            Confidence Evaluation
                         │
          ┌──────────────┴──────────────┐
          │                             │
          ▼                             ▼
 High Confidence                 Low Confidence
          │                             │
          ▼                             ▼
 Continue                  Query Rewriter (Phi-3)
                                        │
                                        ▼
                              Rewrite Validator
                                        │
                                        ▼
                              Rewrite Cache Check
                                        │
                                        ▼
                              Second Retrieval
                                        │
                                        ▼
                              Quality Comparison
                                        │
                         ┌──────────────┴─────────────┐
                         ▼                            ▼
                 Better Retrieval              Worse Retrieval
                         │                            │
                         ▼                            ▼
                 Use New Chunks             Keep Original Chunks

Only validated improvements are accepted, ensuring retrieval quality never degrades.

---

🛡 Grounding & Evidence Validation

Generating an answer is not sufficient; the answer must also be supported by retrieved evidence.

Before returning a response, the system validates whether the retrieved context actually contains sufficient information to answer the question.

Possible grounding outcomes:

Status	Meaning
PASSED	Sufficient evidence exists
REJECTED	Retrieved evidence is insufficient

If grounding fails, the system returns a transparent response rather than hallucinating unsupported information.

---

📑 JSON Response Format

The backend returns structured responses in JSON.

Example:

{
  "answer": "...",
  "grounding_status": "PASSED",
  "constraint_result": "NOT_APPLICABLE",
  "citations": [
      "finance_policy_3"
  ],
  "execution_time_ms": 2943,
  "initialization_time_ms": 0.01
}

---

📈 Performance Summary

The backend was evaluated using both automated benchmark suites and manual acceptance testing.

Automated Validation

Metric	Result
Benchmark Queries	340+
Regression Tests	100% Pass
Acceptance Suite	100% Pass
Rewrite Validator Tests	29/29 Pass

---

Manual Validation

A final manual evaluation was conducted using representative enterprise queries covering:

• Factual retrieval
• Numeric reasoning
• Boolean questions
• Summary generation
• Procedure extraction
• Comparison
• Multi-document reasoning
• Out-of-scope detection

Result

15 / 15 Queries Passed

---

⚡ Performance Optimizations

Several optimizations were introduced to balance retrieval quality with runtime efficiency.

Adaptive Query Rewriting

Only low-confidence retrievals trigger semantic rewriting.

---

Rewrite Validation

Rejects semantic drift before retrieval.

---

LRU Rewrite Cache

Avoids repeated Phi-3 inference for identical queries.

---

Feature Flags

Every Backend v2.0 enhancement is protected behind configuration flags.

ENABLE_QUERY_REWRITER = True

Rollback is immediate by disabling the feature.

---

🧪 Testing Strategy

Testing was performed at multiple levels.

Unit Tests

• Query Rewriter
• Rewrite Validator
• Rewrite Cache
• Adaptive Retrieval

---

Regression Tests

• Preprocessing
• Retrieval
• Empty Queries
• Final Verification

---

Acceptance Tests

Realistic enterprise policy questions covering every supported intent.

---

Stress Tests

• Long queries
• Empty input
• Malformed characters
• Out-of-scope requests
• Timeout recovery
• Ollama unavailable
• Cache failures

---

🎯 Engineering Decisions

Several important architectural decisions shaped the final system.

Why Modular Pipelines?

Each query type requires different retrieval and reasoning behaviour. A modular architecture allows independent optimisation of each pipeline.

---

Why Local Phi-3?

Using Phi-3 locally preserves Gemini API quota while enabling semantic query rewriting entirely offline.

---

Why Confidence-Based Rewriting?

Most queries already retrieve correct results.

Running an LLM for every query would unnecessarily increase latency.

Adaptive retrieval ensures rewriting occurs only when beneficial.

---

Why Rewrite Validation?

LLMs occasionally alter numbers, entities, or logical meaning.

The Rewrite Validator prevents such semantic drift before retrieval.

---

Why Rewrite Cache?

Repeated enterprise questions are common.

Caching validated rewrites reduces latency and eliminates redundant local inference.

---

🚧 Current Limitations

Current limitations include:

• Dense vector retrieval only (no hybrid BM25 retrieval)
• CPU inference latency for local Phi-3
• No web interface (backend only)
• No authentication layer
• Single-session document workspace

---

🛣 Future Roadmap

Backend v2.1

• Hybrid Retrieval (Dense + BM25)
• Cross-Encoder Reranking
• Better telemetry dashboard

---

Backend v3

• Agentic Retrieval
• Tool-calling workflows
• Multi-agent reasoning
• Knowledge graph integration

---

Frontend

• Streamlit / React interface
• Drag-and-drop document upload
• Citation viewer
• Interactive retrieval visualization
• Chat history

---

🤝 Acknowledgements

This project was developed as part of an internship focused on building a modular Retrieval-Augmented Generation system using modern NLP techniques.

It combines dense retrieval, adaptive query rewriting, evidence validation, and Model Context Protocol (MCP) integration into a scalable and extensible architecture.

---

📜 License

This project is intended for educational and research purposes.

---

👩‍💻 Author

Nafisa Hasan

B.Tech Computer Science Engineering

Special interests:

• Natural Language Processing
• Retrieval-Augmented Generation
• Large Language Models
• Information Retrieval
• Applied Machine Learning

---

⭐ Final Remarks

This project demonstrates how a traditional Retrieval-Augmented Generation pipeline can be transformed into a modular, intent-aware architecture capable of handling diverse enterprise document question-answering tasks with improved retrieval quality, grounded responses, and extensible design.

Rather than relying on a single retrieval strategy, the system incorporates adaptive retrieval, semantic query rewriting, rule-based validation, evidence grounding, and modular routing to improve both reliability and maintainability.

The resulting architecture serves as a strong foundation for future work in hybrid retrieval, agentic workflows, and production-scale document intelligence systems.

---

"Good retrieval finds information. Great retrieval understands the question first."