MCP Chat

Note: This project is an extension of the MCP certification project originally sourced from the Anthropic Claude MCP course. It has been extended to support local LLMs instead of the Anthropic API, along with additional tooling and improvements.

A command-line interface application that enables interactive chat with a local LLM (via Ollama or any OpenAI-compatible server) using the MCP (Model Context Protocol) architecture for document management.

---

Prerequisites

• Python 3.9+
• Ollama or any OpenAI-compatible local LLM server

---

Setup

Step 1: Configure environment variables

Create a .env file in the project root:

LOCAL_LLM_MODEL=llama3.2
LOCAL_LLM_BASE_URL=http://localhost:11434/v1
USE_UV=1   # Set to 0 if not using uv

Step 2: Install dependencies

Option 1: With uv (Recommended)

uv is a fast Python package installer and resolver.

pip install uv
uv venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate
uv pip install -e .

Option 2: Without uv

python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate
pip install openai python-dotenv prompt-toolkit "mcp[cli]==1.8.0"

Step 3: Start your local LLM

# Ollama example
ollama serve
ollama pull llama3.2

Step 4: Run the project

# With uv
uv run main.py

# With additional MCP servers
uv run main.py extra_server.py another_server.py

---

Usage

Basic Chat

Type your message and press Enter:

> What is the state of the condenser tower?

Document Retrieval

Use @ followed by a document ID to include its contents in your query:

> Tell me about @deposition.md
> Summarize @financials.docx

Demo

MCP Chat running with llama3.2 via Ollama on Windows

Commands

Use / prefix to execute MCP prompts. Press Tab to autocomplete:

> /format deposition.md
> /summarize report.pdf

---

Available Documents

Document	Description
deposition.md	Testimony of Angela Smith, P.E.
report.pdf	State of a 20m condenser tower
financials.docx	Project budget and expenditures
outlook.pdf	Projected future performance
plan.md	Project implementation steps
spec.txt	Technical equipment requirements

---

Architecture

main.py
  ├── MCPClient          # Manages stdio communication with MCP server(s)
  ├── mcp_server.py      # FastMCP server — tools, resources, prompts
  ├── core/claude.py     # Local LLM integration (OpenAI-compatible)
  ├── core/cli_chat.py   # Chat logic with @ and / command handling
  └── core/cli.py        # Terminal UI with Tab autocomplete

---

MCP Server Features

Feature	Name	Description
Tool	read_doc_contents	Read the contents of a document by ID
Tool	edit_document	Replace text within a document
Resource	docs://documents	List all available document IDs
Resource	docs://documents/{doc_id}	Fetch contents of a specific document
Prompt	/format	Rewrite a document in Markdown format

---

Development

Adding New Documents

Edit the docs dictionary in mcp_server.py:

docs = {
    "your_doc.md": "Your document content here",
}

Adding New MCP Servers

Pass additional server scripts as arguments when running:

uv run main.py your_custom_server.py

Adding New Tools / Prompts / Resources

Use the FastMCP decorators in mcp_server.py:

@mcp.tool(name="my_tool", description="Does something useful")
def my_tool(input: str) -> str:
    return f"Processed: {input}"

---

Known Limitations

• Document edits are in-memory only and lost on server restart
• No persistent storage backend
• No authentication or multi-user support (stdio only — single client per server instance)

---

Troubleshooting

OneDrive hardlink error on Windows:

$env:UV_LINK_MODE = "copy"; uv pip install -e .

Local LLM not responding:

• Ensure Ollama is running: ollama serve
• Confirm the model is pulled: ollama pull llama3.2
• Check LOCAL_LLM_BASE_URL in .env matches your server

Module not found errors:

uv add openai python-dotenv mcp