OnMind-RAG

A complementary and independent project from OnMind-PUB: a lightweight knowledge base (light RAG) that exposes an MCP server so an agent (MCP client) can query the content of one or more sites.

It is not part of the init → index → build → publish workflow. It only reads the _index.json already generated by a site and, on demand, the markdown.

Idea

1. On startup, the server loads docs/public/_index.json from each configured site.
2. Normalizes in memory (stable IDs, clean tags, visibility, path to .md).
3. Body cache: reads each .md once, stores bodyText (no frontmatter) and logs time/size to stderr.
4. Search engine: Orama BM25 full-text by default; hybrid/vector modes opt-in with embeddings.
5. Exposes compact MCP tools: summary → search (meta + body) → document reading.

No _catalog.json is written. If the site re-indexes externally, use the reload_catalog tool.

Requirements

• Node.js ≥ 18 (or Bun with Node compatibility)
• At least one site with docs/public/_index.json (e.g. sites/know from OnMind-PUB)

Setup

cd rag
cp .env.example .env   # optional; you can also export vars
bun install            # or: npm install

Variables (see .env.example):

Variable	Meaning
RAG_SITES	Paths to site roots, comma-separated (required)
RAG_SITE_NAMES	Optional labels (same order)
RAG_VISIBILITY	public (default) | protected | all
RAG_MAX_BODY	Max chars returned by read_document (default 50000; increase for very long docs)
RAG_CACHE_BODY	1 (default) caches body at startup → Orama indexes body → full-text on markdown. 0 = metadata only (title/desc/tags) → no body search; read_document falls back to disk.
RAG_SEARCH_MODE	fulltext (default, BM25) | hybrid | vector (last two require RAG_EMBEDDINGS=1)
RAG_EMBEDDINGS	1 enables local hash embeddings for hybrid/vector; 0 (default)
RAG_EMBED_DIMS	Vector dimensions (default 384)
RAG_SNAPSHOT	1 saves/loads Orama snapshot to disk; 0 (default)
RAG_SNAPSHOT_PATH	Snapshot path (default ./data/orama-snapshot.json)

Example:

export RAG_SITES=../sites/know
# or multiple:
# export RAG_SITES=../sites/know,../sites/andrey

Smoke test (no MCP)

RAG_SITES=../sites/know bun run smoke

With embeddings + hybrid:

RAG_SITES=../sites/know RAG_EMBEDDINGS=1 RAG_SEARCH_MODE=hybrid bun run smoke

With snapshot (2nd load ~100 ms):

RAG_SITES=../sites/know RAG_SNAPSHOT=1 bun run smoke   # first: saved
RAG_SITES=../sites/know RAG_SNAPSHOT=1 bun run smoke   # second: loaded

Start MCP server (stdio)

RAG_SITES=../sites/know bun run mcp

MCP Client (Cursor / Claude Desktop / Grok / Jan)

{
  "mcpServers": {
    "onmind-rag": {
      "command": "bun",
      "args": ["/absolute/path/to/pub/rag/src/server.js"],
      "env": {
        "RAG_SITES": "/absolute/path/to/pub/sites/know",
        "RAG_VISIBILITY": "public",
        "RAG_SEARCH_MODE": "fulltext",
        "RAG_EMBEDDINGS": "0",
        "RAG_SNAPSHOT": "0"
      }
    }
  }
}

You can also use node instead of bun

{
  "command": "node",
  "args": ["/absolute/path/to/pub/rag/src/server.js"],
  "env": { "RAG_SITES": "/absolute/path/to/pub/sites/know" }
}

Tools

Tool	Purpose
list_sites	Loaded sites, body-cache stats, search-engine status (Orama, embeddings, snapshot)
catalog_summary	Stats by category / language / tags (orientation)
search_content	Full-text Orama (BM25) by default on title/description/tags/body. mode: fulltext|hybrid|vector optional. Returns cards with match.score and match.snippet.
get_entry	One record by id (site:url)
read_document	Markdown body (preferably from cache; truncatable)
list_series	Series ordered by filename within a category
reload_catalog	Re-reads _index.json + body cache + rebuilds index

Typical agent flow:

catalog_summary → search_content → get_entry / list_series → read_document

Relation to OnMind-PUB

OnMind-PUB	onmind-rag
Generates _index.json and the static site	Only consumes it
Publish workflow	Outside that flow
Lives in monorepo for convenience	Movable: just point RAG_SITES to any folder with docs/public/_index.json

Body cache (startup)

In stderr you'll see something like:

[onmind-rag] index: 289 entries from 1 site(s) in 3ms at ...
[onmind-rag] body cache: 289 docs, 3533.3 KiB in 59ms (missing path=0, read errors=0)
[onmind-rag] search engine: orama · mode=fulltext · embeddings=off · 289 docs · 450ms

With RAG_CACHE_BODY=1 (default): Orama indexes the body field → search_content performs full-text BM25 over the entire markdown. read_document answers from memory (fromCache: true).

With RAG_CACHE_BODY=0: Orama only indexes metadata (title, description, tags) → search_content does not find matches in the document body. read_document falls back to disk reads.

Disable only if RAM is very tight or corpus > 50 MB and you accept metadata-only search.

Search engine (Orama)

Mode	What it does
fulltext (default)	BM25 with stemming, typo tolerance, field boosting. Fast, no external deps.
hybrid	BM25 + vector (cosine) — needs RAG_EMBEDDINGS=1
vector	Vector similarity only — needs RAG_EMBEDDINGS=1

Embeddings (opt-in)

RAG_EMBEDDINGS=1 uses local feature-hashing (no TensorFlow, no API keys) — a portable baseline to exercise the vector/hybrid path. Not a SOTA semantic model. For real quality, replace embed.js with a provider (OpenAI, Xenova/transformers.js, etc.) keeping the embedText(text, { dims }) interface.

Snapshot (opt-in)

RAG_SNAPSHOT=1 serializes the Orama index to disk (data/orama-snapshot.json + .meta.json). On subsequent starts, if the corpus fingerprint (ids, titles, body length, hide, embeddings flag/dims) matches, it loads the snapshot in ~100 ms instead of re-indexing (~450 ms).

Limits (by design)

• Lexical/BM25 retrieval + hash vectors (no SOTA embeddings unless you plug them in).
• No knowledge graph / neighbors yet (next step: markdown links).
• Designed as a useful, portable base — not a managed vector engine.

Why Spanish tokenizer?

The corpus is predominantly Spanish (~80%). Orama's default English tokenizer treats short technical codes like "XDB", "IAM", "UCDM" as stop-words/noise. Setting language: 'spanish' in the tokenizer indexes them correctly.

---

Based on onmind-rag v0.2.0 — MCP server for knowledge retrieval with Orama BM25 + opt-in hybrid/vector + snapshot