English | 中文

wandering-rag-mcp

A local RAG (Retrieval-Augmented Generation) knowledge base MCP server that exposes semantic document search as tools. Uses zvec (Alibaba's embedded vector database) for vector storage and Qwen3-Embedding-0.6B for text embedding.

No external LLM required — the MCP server handles retrieval, and the client (QoderWork, Claude Desktop, etc.) provides generation.

Features

• Multi-format support: Plain text files (40+ types: md, txt, py, js, ts, go, rs, etc.) and binary documents (PDF, DOCX, PPTX, XLSX)
• Embedded vector DB: zvec — zero-config, no Docker, WAL-persistent, HNSW-indexed
• Local embedding: Qwen3-Embedding-0.6B (0.6B params, 1024-dim, 32K context, bilingual CN/EN)
• Optional reranker: bge-reranker-v2-m3 cross-encoder for higher retrieval accuracy
• REST API: HTTP endpoints for document management (upload/search/delete), runs alongside MCP on the same port
• Three transport modes: stdio, SSE, Streamable HTTP
• Multi-collection: Isolate documents into separate knowledge bases

Quick Start

Prerequisites

• Python >= 3.10

Install

git clone <repo-url>
cd wandering-rag-mcp
pip install -e .

Run

# stdio mode (default, for QoderWork / Claude Desktop)
python server.py

# SSE mode
python server.py --mode sse --port 8000

# Streamable HTTP mode
python server.py --mode streamable-http --host 0.0.0.0 --port 8000

# Disable REST API (MCP only)
python server.py --mode sse --no-api

Environment variables are also supported:

Variable	Description	Default
RAG_MCP_MODE	Transport mode	stdio
RAG_MCP_HOST	Bind host	127.0.0.1
RAG_MCP_PORT	Bind port	8000
RAG_EMBEDDING_MODEL	Embedding model name	Qwen/Qwen3-Embedding-0.6B
RAG_RERANKER_MODEL	Reranker model name	BAAI/bge-reranker-v2-m3
RAG_DATA_DIR	Vector data directory	./data
RAG_CORS_ORIGINS	Allowed CORS origins (comma-separated)	*

Client Configuration

stdio Mode (QoderWork / Claude Desktop)

{
  "mcpServers": {
    "wandering-rag-mcp": {
      "command": "python",
      "args": ["D:\\repos\\rag-mcp\\server.py"]
    }
  }
}

SSE Mode

{
  "mcpServers": {
    "wandering-rag-mcp": {
      "url": "http://your-server:8000/sse"
    }
  }
}

Streamable HTTP Mode

{
  "mcpServers": {
    "wandering-rag-mcp": {
      "url": "http://your-server:8000/mcp"
    }
  }
}

MCP Tools

search

Search the knowledge base with natural language queries.

Parameter	Type	Default	Description
query	string	(required)	Natural language search query
top_k	int	5	Number of results to return
collection	string	"default"	Collection to search
rerank	bool	false	Use cross-encoder reranker for higher accuracy
filter	string	""	Glob pattern to filter by source file (e.g. *.md, **/docs/*)
expand_context	int	0	Number of neighboring chunks to include before/after each result for broader context

ingest_file

Import a single file into the knowledge base.

Parameter	Type	Default	Description
filepath	string	(required)	Path to the file
collection	string	"default"	Target collection
chunk_size	int	500	Max characters per chunk
force	bool	false	Re-import even if file hasn't changed
chunk_mode	string	"recursive"	Chunking strategy: recursive (character-based splitting), semantic (embedding similarity-based splitting), or structural (document structure-aware splitting by headings, code blocks, tables)

Change detection: By default, files that haven't changed since last import are skipped. Use force=true to re-import anyway.

Supported formats: .md, .txt, .py, .js, .ts, .pdf, .docx, .pptx, .xlsx, and 40+ more.

ingest_directory

Batch import all files in a directory.

Parameter	Type	Default	Description
dirpath	string	(required)	Directory path
collection	string	"default"	Target collection
recursive	bool	true	Scan subdirectories
extensions	string	""	Comma-separated extensions filter (empty = all supported)
chunk_size	int	500	Max characters per chunk
force	bool	false	Re-import even if files haven't changed
chunk_mode	string	"recursive"	Chunking strategy: recursive, semantic, or structural

list_collections

List all knowledge base collections.

list_documents

List all documents in a collection.

Parameter	Type	Default	Description
collection	string	"default"	Collection name

delete_document

Remove a document and all its chunks from the knowledge base.

Parameter	Type	Default	Description
filepath	string	(required)	Path used during import
collection	string	"default"	Collection name

delete_collection

Delete an entire knowledge base collection and all its documents, vectors, and configuration. This cannot be undone.

Parameter	Type	Default	Description
collection	string	"default"	Collection name to delete

configure_collection

Set default parameters for a knowledge base collection. Future import and search operations will use these defaults when parameters are not explicitly specified.

Parameter	Type	Default	Description
collection	string	"default"	Collection name
chunk_mode	string	""	Default chunking strategy. Empty = keep current. recursive, semantic, or structural
chunk_size	int	0	Default max characters per chunk. 0 = keep current
chunk_overlap	int	-1	Default overlap characters. -1 = keep current
rerank	bool	None	Default whether to use reranker for search. None = keep current
description	string	None	Collection description. None = keep current

get_collection_config

View the current configuration for a collection.

Parameter	Type	Default	Description
collection	string	"default"	Collection name

REST API

When running in SSE or Streamable HTTP mode, a REST API is automatically available at /api/ alongside the MCP endpoint. This enables web frontends (e.g., CodingHub) to manage documents via HTTP while AI clients use MCP for search.

Disable with --no-api if you only need MCP.

GET /api/health

Health check endpoint.

GET /api/collections

List all knowledge base collections.

Response:

[{"name": "default", "doc_count": 5}]

GET /api/collections/{name}/documents

List all documents in a collection.

Response:

[{"source": "/path/to/file.md", "chunk_count": 12}]

POST /api/collections/{name}/documents

Upload a file to the knowledge base. Accepts multipart/form-data with a file field.

curl -F "file=@document.pdf" http://localhost:8000/api/collections/default/documents

Optional query parameters: chunk_size (default: 500), chunk_mode (recursive, semantic, or structural, default: recursive).

Response:

{"status": "ok", "filename": "document.pdf", "chunks": 24}

DELETE /api/collections/{name}/documents

Delete a document and all its chunks.

curl -X DELETE http://localhost:8000/api/collections/default/documents \
  -H "Content-Type: application/json" \
  -d '{"filepath": "/path/to/file.md"}'

Response:

{"status": "ok", "filepath": "/path/to/file.md", "deleted": 12}

DELETE /api/collections/{name}

Delete an entire collection and all its data.

curl -X DELETE http://localhost:8000/api/collections/my_collection

Response:

{"status": "ok", "collection": "my_collection", "deleted": true}

POST /api/collections/{name}/search

Semantic search across the knowledge base.

curl -X POST http://localhost:8000/api/collections/default/search \
  -H "Content-Type: application/json" \
  -d '{"query": "how to install", "top_k": 5, "rerank": false, "filter": "*.md", "expand_context": 1}'

Request body:

Field	Type	Default	Description
query	string	(required)	Search query
top_k	int	5	Number of results
rerank	bool	false	Use cross-encoder reranker
filter	string	""	Glob pattern to filter by source file path
expand_context	int	0	Number of neighboring chunks to include before/after each result

Response:

[
  {"id": "...", "score": 0.85, "text": "...", "source": "file.md", "chunk_index": 3}
]

GET /api/collections/{name}/config

Get the configuration for a collection.

Response:

{"chunk_mode": "semantic", "chunk_size": 500, "chunk_overlap": 50, "rerank": false, "description": "Technical docs"}

PUT /api/collections/{name}/config

Update collection configuration. Only include fields you want to change.

curl -X PUT http://localhost:8000/api/collections/default/config \
  -H "Content-Type: application/json" \
  -d '{"chunk_mode": "semantic", "description": "Technical documentation"}'

Response: Returns the full updated configuration.

CORS

The REST API includes CORS headers by default (allows all origins). Restrict with the RAG_CORS_ORIGINS environment variable:

RAG_CORS_ORIGINS=http://localhost:5173,http://localhost:8080 python server.py --mode sse

Architecture

flowchart TB
    subgraph Client["MCP Client (QoderWork, etc.)"]
        direction LR
        C1["User question"] --> C2["Call MCP tools"] --> C3["LLM answer"]
    end

    Client <-->|"stdio / SSE / Streamable HTTP"| Server

    subgraph Server["RAG MCP Server (FastMCP)"]
        direction LR
        subgraph Tools[" "]
            direction TB
            T1["Ingest Pipeline"] ~~~ T2["Search Pipeline"] ~~~ T3["Collection Manager"]
        end
        Tools --> Embed & Rerank & Vec
        Embed["sentence-transformers<br/>Qwen3-Embedding-0.6B"]
        Rerank["Cross-Encoder<br/>bge-reranker-v2-m3"]
        Vec["zvec<br/>./data/"]
    end

    style Client fill:#e8f4f8,stroke:#2196F3
    style Server fill:#f5f5f5,stroke:#333
    style Tools fill:#fff3e0,stroke:#FF9800
    style Embed fill:#fce4ec,stroke:#E91E63
    style Rerank fill:#e8eaf6,stroke:#3F51B5
    style Vec fill:#f3e5f5,stroke:#9C27B0

Project Structure

wandering-rag-mcp/
├── pyproject.toml          # Dependencies and entry point
├── server.py               # MCP server entry + 6 tool definitions + combined ASGI
├── api/
│   ├── __init__.py
│   └── app.py              # REST API routes (starlette)
├── core/
│   ├── chunker.py          # Text chunking (recursive + semantic)
│   ├── embeddings.py       # sentence-transformers wrapper (lazy load)
│   ├── reranker.py         # Cross-encoder reranker (lazy load)
│   ├── service.py          # Shared business logic (MCP + REST)
│   └── vector_store.py     # zvec wrapper (CRUD + search)
├── data/                   # zvec storage (auto-created at runtime)
│   └── default/
└── .gitignore

How It Works

1. Ingest: File is read (plain text or converted via markitdown) → split into overlapping chunks → each chunk embedded into a 1024-dim vector → stored in zvec with metadata (text, source path, chunk index)

2. Search: Query text → embedded into vector → zvec ANN search returns top-k nearest chunks with similarity scores → optionally reranked by cross-encoder for higher accuracy → returned as formatted text with source references

3. Document ID: SHA256 hash of the file path (first 16 chars) is used as a stable document ID, enabling idempotent re-imports and deletion by file path.

Dependencies

Package	Purpose
mcp	MCP protocol SDK (FastMCP)
zvec	Embedded vector database by Alibaba
sentence-transformers	Load and run embedding models
markitdown[all]	Convert PDF/DOCX/PPTX/XLSX to Markdown
python-multipart	Multipart form parsing for REST API file uploads

Technical Documentation

For detailed architecture and technical stack explanation, see Architecture Document.

Deployment

Quick Install (Online)

For a clean Linux server with internet access:

curl -sSL https://raw.githubusercontent.com/mambo-wang/wandering-rag-mcp/main/deploy/setup.sh | bash

This installs everything: Python venv, dependencies, embedding model, and generates start scripts.

Offline Install

For air-gapped servers, use the offline packaging scripts in deploy/:

# On a machine with internet: prepare the bundle (~3GB with models)
cd deploy && bash prepare.sh x86_64

# Transfer wandering-rag-mcp-offline.tar.gz to the target server, then:
tar xzf wandering-rag-mcp-offline.tar.gz
cd bundle && bash install.sh

See deploy/README.md for full deployment guide.

Roadmap

The following improvements are planned for future releases:

• Hybrid search: Combine BM25 keyword retrieval with semantic search using Reciprocal Rank Fusion (RRF) for better precision on exact-match queries (function names, error codes, technical terms)
• SQLite metadata layer: Replace _registry.json with SQLite for document metadata, enabling server-side metadata filtering (WHERE clauses) and reliable batch deletion instead of the current ID-probing approach
• Evaluation framework: Built-in recall@k and MRR benchmarks with a CLI evaluation script, enabling quantitative measurement of retrieval quality when tuning chunking strategies or swapping models
• Token-based chunk sizing: Replace character-based chunk_size with token-based sizing for consistent chunk lengths across different languages (CJK vs. Latin scripts)
• Embedding batch control: Configurable batch size for encode() to prevent memory spikes when ingesting large documents with hundreds of chunks
• Concurrent access safety: File-level locking for _registry.json and thread-safe VectorStore operations to prevent corruption under concurrent REST API requests

License

MIT