Local MLX MCP

An MCP server that turns three locally hosted models into named tools inside Claude Desktop and Codex.

The models run on a Mac mini using MLX. The bridge exposes them over the LAN, so a laptop on the same network can call a local model as a tool without installing anything or shipping the prompt to a hosted API.

Tools

Tool	What it does
list_local_models	Lists the models the host is currently serving.
ask_llama	Llama 3.2 3B. Fast, for quick passes.
ask_qwen	Qwen 3.5 4B. Reasoning and code.
ask_gemma	Gemma 4 E2B. General purpose.

Any of the ask_* tools can optionally run a SearXNG search first and pass the results to the model, so a local model can answer questions about things that happened after its training cutoff.

Why bother

Two reasons, and neither is cost.

The first is data. Some documents cannot go to a hosted API, and "cannot" is a policy fact rather than a preference. Having a local model available as a tool means the sensitive step of a workflow can run locally while the rest of it runs wherever it likes.

The second is that routing is a real design decision. Not every step needs a frontier model. Being able to name a small local model as the worker for a cheap step, in the same interface where the expensive steps run, makes that decision explicit instead of accidental.

Running it

Over HTTP, for LAN clients:

node server/mlx-bridge.mjs
# listens on 0.0.0.0:8090

Over stdio, for a local client:

node server/mlx-bridge.mjs --stdio

Point it at the host serving the models:

MLX_HOST=http://127.0.0.1   # or the host's LAN address
MCP_PORT=8090

The upstream model servers are OpenAI-compatible, so the bridge speaks ordinary chat-completions to them on ports 8000, 8001, and 8002.

Claude Desktop extension

claude-extension/ packages the same server as a desktop extension with a manifest declaring the four tools. Install it and the models appear as tools without editing a config file by hand.

Wiring it into a client config

Copy mcp.json.example, replace the placeholder paths with absolute paths, and drop it into your client's MCP config location. The example also includes the SearXNG server, since the two are normally used together.

Running the models at boot

launchd/ holds the macOS agents that keep the model servers and the bridge running: one per model plus one for the bridge itself. Paths in these files are placeholders and need to be pointed at your install before loading them with launchctl.

Requirements

• Node 18 or newer.
• A host serving MLX models on the expected ports.
• Optional: a SearXNG instance for the search-augmented calls.

Memory is the real constraint. On a 16 GB machine, running three MLX models, an embedding model, and Ollama at once does not work. Pick which ones stay loaded.

Licence

MIT.