lucidmem

A memory engine for AI agents, with per-project isolation that actually holds, a dashboard to manage what's in it, and an MCP connector so agents can use it directly.

Agents forget everything between sessions. The usual fix is a vector store, and the usual problem is that a vector store has no idea who is asking — so the moment more than one person or project uses it, everyone sees everything. lucidmem makes the project (a space) the unit of access, and enforces it in Postgres rather than in application code.

Space ──┬── Document ──── Memory (chunk)
        └── Fact (versioned; a contradiction supersedes its parent)

• Two layers. A curated layer of hand-written Markdown in git (OKF v0.2), and a corpus layer of ingested material. Curated outranks corpus, and the UI and API label which is which — because "someone wrote this down on purpose" and "this appeared in a log once" deserve different trust.
• Explainable ranking. Every result decomposes its own score into vector, lexical, recency, curated and rerank contributions. explain_ranking compares any two results.
• Degrades honestly. GPU services are optional. When they are gone, search still works and says rerank: unavailable rather than silently changing.
• Multi-tenant by construction. Row-level security is the boundary; application filtering is only the fast path.

Install

git clone https://github.com/libr3andr3/lucidmem && cd lucidmem
cp deploy/env.example deploy/.env      # set POSTGRES_PASSWORD and KB_APP_PASSWORD
docker compose -f deploy/compose.yml --env-file deploy/.env up -d

Then create a project and a token:

kb init myproject          # scaffolds a bundle, kb.yaml, .mcp.json, an agent skill
kb apply                   # reconcile grants (prints a diff first)
kb token issue user:you

Open http://127.0.0.1:8090 and paste the token.

For just the CLI and MCP server:

pip install -e .

Connect an agent

docs/CONNECT.md is the full guide. The short version:

{
  "mcpServers": {
    "lucidmem": {
      "command": "python",
      "args": ["-m", "lucidmem.mcp_server"],
      "env": { "LUCIDMEM_API_URL": "https://kb.example.com" }
    }
  }
}

Tools: find_memory, drill_down, save_memory, ingest_file, promote, explain_ranking, fact_history, memory_stats, whoami.

Access control

Concept	Meaning
space	A project. The unit of isolation.
principal	A user, agent or service. Holds tokens.
grant	principal-or-group × space × role
role	reader → contributor → curator → admin

Grants live in a kb.yaml per project, so "who can read this?" is a reviewable diff instead of tribal knowledge:

space: sales
title: Sales
sources:
  - {path: ./contracts, parser: pdf, tags: [legal]}
groups:
  bizdev: ["user:ana", "user:ben"]
grants:
  - {principal: "user:ana", role: admin}
  - {group: bizdev, role: contributor}

kb apply reconciles it and asks before revoking anything.

The boundary is Postgres RLS. kb-api connects as a role that cannot bypass it and sets the principal per transaction. Clients get a bearer token, never a DSN. tests/test_isolation.py includes a test that bypasses the application entirely and queries the table directly.

Ingest

Parser	Handles
transcript	Claude Code session .jsonl — keeps prompts, prose, commands, short results; drops thinking and file dumps (~1% of bytes on real data)
pdf	Text layer, else OCR, else a VLM description for pages that are one figure
image	Description plus transcription
okf	An OKF bundle — the curated layer
text	Markdown, code, config

Credentials are stripped before anything is embedded or sent to a model, so a secret pasted into a terminal two months ago never becomes a searchable memory. Ingestion is a queue: GPU-backed work waits when the GPU is away instead of failing.

Optional GPU services

All optional, all degrade cleanly. deploy/serve.sbatch is an example SLURM job.

Service	Used for	Without it
VLM (any OpenAI-compatible endpoint)	contextual-retrieval prefixes, fact extraction, figure description	prefixes skipped; chunks embed bare
OCR	scanned documents	those files stay queued
Embeddings (GPU)	bulk ingest	falls back to CPU (~15s/chunk vs ~0.1s)
Reranker	final ordering	plain RRF, reported in why

The GPU and CPU embedders run the same code so their vectors share a space; tests/test_embed_drift.py asserts mean cosine > 0.99.

Tests

pytest                                            # units, no database
docker run -d --name kb-pgtest -e POSTGRES_PASSWORD=testpw \
  -e POSTGRES_USER=lucidmem -e POSTGRES_DB=lucidmem \
  -p 127.0.0.1:5433:5432 pgvector/pgvector:pg17
LUCIDMEM_TEST_DB=1 pytest                         # + the isolation suite

The isolation suite runs against a deterministic hash embedder, so it needs no GPU. A security test that only runs when a GPU is free is a test that never runs.

Docs

• docs/CONNECT.md — connecting an agent
• docs/ARCHITECTURE.md — how it fits together, and why

License

MIT