local-llm-mcp

English | 日本語

https://github.com/naka-koma/local-llm-mcp

A thin MCP server for delegating lightweight tasks from Claude Code (or any other MCP-compatible client) to local or cloud LLMs via LiteLLM.

Designed as a general-purpose tool with no dependency on any specific project. Set it up once, and you can reuse the same configuration from any project on your machine.

What is this for?

Claude Code's Task tool (the subagent mechanism) only lets you choose between sonnet, opus, and haiku as the model. It isn't designed to let you use other vendors' models (such as local LLMs) directly as subagents.

Instead of "adding more subagents" to work around this limitation, this tool adds "one more tool" to enable delegation to any LLM.

Claude Code ──(MCP)──> local-llm-mcp ──(HTTP)──> LiteLLM ──> Ollama (local) / cloud API

It's intended for offloading lightweight tasks that the main agent doesn't need to handle itself — summarizing test results, pre-reading logs, simple classification, and so on.

Prerequisites

• Python 3.10+
• Ollama running locally (if using a local model)
• LiteLLM running as a proxy

This repository provides only the MCP server itself. Setting up Ollama or LiteLLM is out of scope.

Setup

1. Install dependencies

pip install -r requirements.txt

2. Configure environment variables

Refer to .env.example and set the LiteLLM endpoint and model name (if you use a .env file, you'll need something like python-dotenv to load it separately; setting the variables directly in your environment also works).

Variable	Description	Default
LITELLM_BASE_URL	LiteLLM proxy endpoint	http://localhost:4000
LITELLM_MODEL	The model_name registered in LiteLLM's config.yaml	local-gemma
LITELLM_TIMEOUT_SECONDS	Response timeout (seconds)	30

3. Register with Claude Code

Registering with the -s user scope makes the tool available from any project on this machine.

Run via uvx (no clone needed — for embedding in other projects)

If uv is installed, you can run it directly without cloning the repository. This is the recommended approach when embedding into another project's setup steps (e.g. Flowrite).

claude mcp add local-llm -s user -- uvx --from git+https://github.com/naka-koma/local-llm-mcp local-llm-mcp

On first run, uv automatically builds the environment including dependencies, so pip install isn't needed.

Use the setup script (recommended for local development)

Runs dependency installation and claude mcp add together.

git clone https://github.com/naka-koma/local-llm-mcp.git
cd local-llm-mcp

# macOS / Linux / Git Bash
bash scripts/setup.sh

# Windows PowerShell
pwsh scripts/setup.ps1

Register manually

git clone https://github.com/naka-koma/local-llm-mcp.git
cd local-llm-mcp
pip install -r requirements.txt
claude mcp add local-llm -s user -- python /path/to/local-llm-mcp/server.py

On Windows, if python's PATH is unreliable, we recommend using the py launcher (C:\Windows\py.exe, which has system-wide PATH set up).

claude mcp add local-llm -s user -- py D:\path\to\local-llm-mcp\server.py

Provided tool

ask_local_llm(prompt: str) -> str

Sends a prompt to an LLM via LiteLLM and returns the response text. On connection errors or timeouts, it doesn't raise an exception — it returns a string describing the error instead.

How to use it in your project

We recommend adding a conditional instruction like the following to the consuming project's CLAUDE.md (or equivalent):

## Using a local LLM

If an MCP tool called `local-llm` is available, lightweight tasks such as
summarizing test results or pre-reading logs may be delegated to it.

Even in an environment where the tool doesn't exist, this has no impact
on this project's behavior (delegation is "use it if it's there," not a
required dependency).

License

MIT License. See LICENSE for details.