MailOKF MCP

MailOKF is a local-first Python MCP server that turns Gmail, Microsoft Outlook/Microsoft 365, iCloud Mail, and generic IMAP/SMTP mailboxes into a synchronized, searchable knowledge layer—without making the local copy authoritative.

It exposes 38 tools and four resources for mailbox actions, synchronization, OKF retrieval, attachments, and optional semantic search. The provider mailbox remains authoritative; SQLite and OKF are local projections.

MailOKF combines full email operations with durable Open Knowledge Format (OKF) projections in one local-first MCP:

• full initial mailbox import;
• provider-specific incremental synchronization;
• normalized SQLite storage and FTS5 search;
• one Open Knowledge Format (OKF) source concept per email;
• local, content-addressed attachment retention;
• raw MIME deletion after successful conversion;
• optional CocoIndex + sqlite-vec semantic indexing;
• draft, send, reply, forward, move, delete, label/category, and batch tools;
• explicit confirmation and idempotency controls for outbound email.
• tamper-evident managed outbound attachments for new sends and drafts.

Implementation status: MailOKF 0.3 provides 25 stable core email operations with camelCase calling shapes plus thirteen MailOKF extensions. Live Gmail, Microsoft, and iCloud interoperability still requires testing with your own provider apps, credentials, tenants, mailbox sizes, throttling conditions, and compliance requirements before production use.

Production notice: This project is local-first software, not a live-provider certification. Test it with isolated accounts and validate provider, tenant, scale, privacy, and compliance requirements before production use.

Keep private: Never commit OAuth JSON, tokens, MAILOKF_ROOT_DIR, runtime mailbox data, attachments, quarantine, or backups.

Documentation

Guide	Use it for
Documentation index	Choose the recommended path for users, operators, or contributors.
Local usage	Install, configure, sync, run MCP, back up, and troubleshoot.
Google Cloud setup	Create a Gmail OAuth desktop client.
Microsoft Entra setup	Register an Outlook public client.
Architecture	Understand authority, storage, synchronization, and failures.
Security	Review supported versions, reporting, secrets, and trust boundaries.
Contributing	Set up development and prepare a pull request.

Outbound attachments

email_send and email_draft_create accept an optional strict attachments array. Both require a nonblank idempotencyKey before provider access; terminal send/draft retries replay durable results and ambiguous provider outcomes remain blocked. Each item must select exactly one source: {stagedId, filename?}, {messageId, attachmentId, filename?}, or {localPath, filename?, confirmed: true}. Reply and forward do not accept attachment inputs.

Use mail_attachment_list(account_id, message_id) to inspect retained message attachments without paths or bytes. Its typed results include messageId and attachmentId, so they can be copied into the retained selector above. mail_attachment_stage(local_path, filename?, confirmed?) returns a typed result whose stagedId can be copied into the staged selector. mail_attachment_create accepts filename, optional content_type, and exactly one tagged content shape: {kind: "text", contentText: "..."} or {kind: "base64", contentBase64: "..."}. Its typed result also exposes stagedId. Configured workspace/import roots are allowed; arbitrary paths require both MAILOKF_DANGEROUSLY_ALLOW_ARBITRARY_ATTACHMENT_PATHS=true and confirmed=true; the former MAILOKF_ALLOW_ARBITRARY_ATTACHMENT_PATHS name is ignored. MAILOKF_ATTACHMENT_IMPORT_ROOTS uses the platform PATH separator.

Outbound requests accept up to 20 files and 157,286,400 bytes combined; the per-file staging limit is Settings.max_attachment_bytes, configured by MAILOKF_MAX_ATTACHMENT_BYTES. Inline UTF-8 text is size-preflighted in bounded chunks before encoding, and strict base64 is size-preflighted then decoded incrementally without exceeding that per-file cap. Bound filenames and parameter-free MIME types are limited to 255 UTF-8 bytes each; managed metadata sidecars are limited to 4,096 bytes. Count, per-file, total, and metadata limit failures use fixed public messages rather than embedding configuration values.

Staging writes one physical outbound/sha256/<digest> blob per SHA-256 and an opaque, restart-stable sidecar reference that persists no source path or source-path hash. Its validated safe filename persists and defaults to the source basename when no filename override is supplied. An unbound staged or created reference atomically binds to the first account that uses it for email_send or email_draft_create. Content and metadata are reverified before each use. Reusing an already-bound batch creates no empty transaction marker; stale markers with no referencing binding are reclaimed under the store lock. Duplicate output names receive deterministic numeric suffixes. Gmail and IMAP preflight a conservative complete-MIME bound before reading managed descriptors, then build MIME with the preserved filename and type. Gmail rejects a fully serialized RFC 2822 MIME message above its provider-specific 36,700,160-byte cap before creating a send/draft operation; the global attachment contract is unchanged. Outlook uses Graph fileAttachment below 3,000,000 bytes and sequential upload sessions from 3,000,000 through 150,000,000 bytes.

Managed publication is atomic and cross-process serialized. Every configured root ancestor is opened from the filesystem anchor without following symlinks. Platforms without the required descriptor-relative filesystem operations or cross-process file locking fail closed rather than falling back to process-local attachment safety.

Outbound attachments specifically require secure POSIX descriptor operations and fcntl locks. They are intentionally unsupported on Windows; ordinary attachment-free mail, synchronization, search, and the rest of MailOKF's Windows surface remain supported. Windows CI treats this feature as an expected fail-closed capability rather than advertising attachment delivery. Tests that exercise POSIX-only no-follow OAuth configuration reads or descriptor-secure storage migration are also skipped on Windows instead of weakening those filesystem guarantees.

Why this is an MCP server rather than only a converter

The provider mailbox remains authoritative. MailOKF keeps a synchronized local projection and exposes both:

• resources and retrieval tools for local OKF knowledge;
• action tools for provider operations such as draft, send, reply, move, and label.

This keeps read-heavy agent queries local and fast, while provider APIs remain available for fresh synchronization and controlled writes.

Architecture

flowchart LR
    G[Gmail API] --> P[Provider adapters]
    O[Microsoft Graph] --> P
    I[iCloud / IMAP + SMTP] --> P
    P --> S[Full + incremental sync engine]
    S --> T[Private raw MIME staging]
    T --> M[MIME parser]
    M --> A[SHA-256 attachment blob store]
    M --> D[SQLite operational DB + FTS5]
    D --> K[Deterministic OKF renderer]
    K --> V[CocoIndex incremental vectors]
    D --> X[MCP tools/resources]
    K --> X
    P --> X

Per-message commit boundary

stage raw MIME privately
  -> parse text and metadata
  -> write/verify attachment blobs
  -> upsert SQLite + FTS
  -> atomically write OKF message and thread files
  -> delete temporary raw MIME unless retention is enabled

If parsing, attachment persistence, database commit, or OKF rendering fails, MailOKF retains the raw message rather than discarding it. It normally moves the file to quarantine; if that move fails, it leaves the bytes in staging and records the fallback path and error.

Storage layout

${MAILOKF_ROOT_DIR:-./lcl_data}/
├── mailokf.json                   # non-secret local layout manifest
├── mailokf.sqlite                 # sync ledger, normalized records, FTS5
├── bundles/
│   └── <account-id>/
│       ├── index.md              # OKF-reserved navigation file
│       ├── log.md                # OKF-reserved date-grouped change log
│       ├── bundle.md             # normal email_knowledge_bundle concept
│       ├── messages/YYYY/MM/*.md # one OKF source concept per message
│       └── threads/*.md          # deterministic thread manifests
├── attachments/
│   └── sha256/ab/cd/<digest>     # deduplicated attachment bytes
├── staging/                       # temporary raw MIME only
├── quarantine/                    # failed raw MIME conversions
├── vectors/
│   ├── okf-vectors.sqlite         # sqlite-vec index
│   └── cocoindex-state/           # CocoIndex incremental state
├── workspace/                     # only authorized outbound attachment import source
├── outbound/                      # private immutable managed outbound attachments
└── config/
    ├── accounts.enc
    ├── master.key
    ├── gateway.token
    └── oauth/

When run from a source checkout without MAILOKF_ROOT_DIR, MailOKF stores all local state in the repository's ignored lcl_data/ directory. Set MAILOKF_ROOT_DIR to use a different portable root; this is required when the application is installed outside a checkout or when a CLI, daemon, gateway, and MCP client must share a separately managed location. Paired MAILOKF_DATA_DIR and MAILOKF_CONFIG_DIR overrides remain available as an explicit legacy split mode. Use mailokf storage show to inspect resolution and mailokf storage consolidate --root <path> after stopping all MailOKF processes to migrate split state safely. To move an already-unified root, keep every MailOKF process stopped and run mailokf storage relocate --source-root <old-root> --root <new-root>; add --move only when verified source removal is intended. Both workflows reject WAL/SHM, links, special files, and overlapping roots, use a private restart journal, verify every destination byte, and publish a newly root-bound mailokf.json only after the destination is complete. Relocation targets one local or cloud-mounted filesystem root and does not implement direct object-storage semantics.

This layout is local-filesystem preparation, not an Azure Blob or S3 implementation. SQLite and its WAL/SHM/lock semantics require a local filesystem and are not object-storage compatible.

Local quick start

Install Python 3.11+ and uv, then use the locked environment:

git clone https://github.com/pat229988/OKF_MAIL.git
cd OKF_MAIL
uv sync --locked
uv run mailokf init
uv run mailokf doctor
uv run mailokf account list

Optional dependencies are explicit:

# Outlook/Microsoft 365 device-code authorization
uv sync --locked --extra outlook

# CocoIndex + sqlite-vec semantic and hybrid search
uv sync --locked --extra vectors

# Both
uv sync --locked --extra outlook --extra vectors

Then follow the Gmail, Outlook, or local IMAP/iCloud setup; test the account; and run an initial sync. CLI, daemon, gateway, and MCP clients must use the same MAILOKF_ROOT_DIR.

Docker Compose local modes

Compose deliberately separates the continuously running synchronization daemon from the client-attached MCP stdio process. Build the image and initialize the unified named root volume:

docker compose build
docker compose run --rm -T mailokf-stdio init

Run the supported long-running polling mode in the background:

docker compose up -d mailokf-daemon
docker compose logs -f mailokf-daemon

Do not run the stdio service detached. Configure an MCP client to create an attached, non-TTY container for each client session (replace the Compose path with the checkout's absolute path):

{
  "mcpServers": {
    "mailokf-docker": {
      "command": "docker",
      "args": [
        "compose",
        "-f",
        "/absolute/path/to/OKF_MAIL/docker-compose.yml",
        "run",
        "--rm",
        "-T",
        "mailokf-stdio"
      ]
    }
  }
}

Compose injects literal policy defaults from .env.example, followed by an optional ignored .env file for local overrides. The container storage root is /mailokf, and both services share one named volume mounted on a directory owned by nonroot UID 10001. The Compose configuration neither publishes an HTTP port nor permits unauthenticated HTTP, even if .env requests it. docker compose down preserves the volumes; adding --volumes permanently removes the local database, retained mail, encrypted account vault, and generated master key.

MCP tool surface

The current contract defines 38 tools (25 compatibility tools plus 13 extensions) and four resource templates. Test sources are maintained under tests/; RELEASE_MANIFEST.json inventories them by path and hash instead of freezing a brittle function or module count here.

Original compatible tools

email_list_accounts       email_add_account          email_remove_account
email_test_account        email_list_folders         email_search
email_get                 email_get_thread           email_get_attachment
email_send                email_reply                email_forward
email_draft_create        email_draft_list           email_move
email_transfer            email_delete               email_mark
email_label               email_folder_create        email_get_labels
email_get_categories      email_batch_delete         email_batch_move
email_batch_mark

MailOKF extensions

mail_sync_initial         mail_sync_now              mail_sync_all
mail_sync_status          okf_validate_bundle        okf_rebuild_bundle
knowledge_search          knowledge_get              vector_index_update
vector_search             mail_attachment_list       mail_attachment_stage
mail_attachment_create

Resources

okf://accounts/{account_id}/index
okf://accounts/{account_id}/messages/{message_id}
okf://accounts/{account_id}/threads/{thread_id}
mailokf://accounts/{account_id}/sync-status

Provider synchronization design

Provider	Initial synchronization	Incremental cursor	Outbound path
Gmail	messages.list + messages.get(format=raw)	historyId + history.list; full resync on expired history	MIME attachments through Gmail drafts/send API
Outlook	folder enumeration + message delta + raw MIME $value	one opaque @odata.deltaLink per folder, immutable IDs	Graph small attachment POST or large upload session on drafts
iCloud / IMAP	folder UID scan + BODY.PEEK[]	UIDVALIDITY, known UID high-water state, periodic reconciliation	MIME attachments through authenticated SMTP and IMAP Drafts

Raw mail and attachments

The default policy is intentionally asymmetric:

• raw .eml files are temporary and deleted only after a successful local commit;
• attachment bytes are retained in a SHA-256 blob store;
• OKF files contain attachment metadata and attachment://sha256/<digest> references;
• normalized text and HTML bodies are stored in SQLite; OKF remains the portable text projection;
• quarantine contains messages that could not be safely converted.

Set MAILOKF_RETAIN_RAW_MIME=true only for debugging or legal-retention requirements and protect that directory accordingly.

Search model

1. FTS5 provides exact and explainable sender, subject, body, and attachment-name retrieval.
2. CocoIndex watches the OKF directory and incrementally reprocesses only changed Markdown files.
3. Sentence Transformers creates local embeddings.
4. sqlite-vec stores and queries vectors in a vec0 virtual table.
5. Hybrid search merges keyword and semantic ranks using reciprocal-rank fusion.

Security boundaries

• Provider credentials are encrypted locally with AES-256-GCM.
• Outbound send/reply/forward tools require confirmed=true and a nonblank idempotency key.
• Pending or unknown provider outcomes block automatic retry with the same key.
• Email contents are untrusted data and never authorize tool calls.
• Attachment paths supplied for outgoing email are validated as local files.
• Raw MIME staging uses private file permissions and quarantine-on-failure semantics.
• Message failures are durably retried and bounded permanent failures are dead-lettered.
• Bare numeric IMAP UIDs require explicit source-folder context.
• Permanent Gmail deletion requires stored https://mail.google.com/ scope; the default setup flow does not request it.
• Separate tenant/account roots must be maintained in multi-user deployments.
• Do not expose streamable HTTP outside localhost without MCP authorization, TLS, and tenant-aware access control.

Development

Use the locked Python 3.11+ environment:

uv sync --locked --extra dev --extra outlook
uv run python -m compileall -q src tests scripts
uv run python scripts/snapshot_mcp_contract.py --check
uv run pytest -q
uv run mailokf --help

See docs/MCP_SURFACE.md for the stable interface map and docs/MCP_CONTRACT.md for contract guarantees.

The test suite covers full and incremental synchronization, remote deletion reconciliation, MIME cleanup, attachment persistence, OKF rendering and validation, FTS5, encrypted credentials, and thread references.

Important production work

Before production rollout, complete provider-specific and operational validation, particularly:

• live provider contract tests;
• OAuth verification/consent and app publishing;
• Graph throttling and large-folder pagination;
• IMAP server compatibility and UID edge cases;
• push notifications/webhooks;
• multi-tenant authorization;
• backup, retention, erasure, and audit policies;
• semantic retrieval evaluation.

Attribution

MailOKF is released under the MIT License. Its stable MCP interface, local-first storage model, and OKF knowledge layer are documented in docs/MCP_SURFACE.md and docs/MCP_CONTRACT.md.