AI Voice Assistant with MCP Tool Calling

A fully free, end-to-end voice-enabled AI assistant built with Python.

Component	Technology
🧠 AI Brain	Google Gemini API (gemini-2.0-flash) — free tier
🎙️ Speech-to-Text	Google Web Speech API via SpeechRecognition — free
🔊 Text-to-Speech	pyttsx3 (offline) — free
🛠️ Tool Calling	Model Context Protocol (MCP) — free
🔍 Web Search	DuckDuckGo — free, no key needed
🌤️ Weather	wttr.in REST API — free, no key needed

---

Features

• 🎙️ Voice Input — speak naturally; the assistant understands you
• 🔊 Voice Output — responses are read aloud via offline TTS
• 🤖 Gemini AI — context-aware, multi-turn conversation
• 🛠️ 7 MCP Tools available to the AI:
  1. Calculator — safe math expression evaluator
  2. Web Search — DuckDuckGo (no API key)
  3. Weather — real-time via wttr.in (no API key)
  4. Date/Time — current date and time
  5. Read File — read any local file
  6. Write File — write/append to a local file
  7. List Directory — browse local folders
• 💬 Multi-turn memory — remembers conversation context
• ⌨️ Text mode — works without a microphone (--text flag)

---

Prerequisites

• Python 3.11+
• A free Gemini API key — get one at https://aistudio.google.com/app/apikey
• Internet connection (for speech recognition, Gemini API, and web tools)
• Microphone (optional — text mode works without one)

---

Installation

1. Clone / download the project

# If using git:
git clone <your-repo-url>
cd "AI voice assistant"

# Or just open the folder in your terminal
cd "C:\Users\sasid\Downloads\AI voice assistant"

2. Create and activate a virtual environment (recommended)

python -m venv .venv

# Windows:
.venv\Scripts\activate

# macOS / Linux:
source .venv/bin/activate

3. Install PyAudio (Windows — required for microphone)

PyAudio on Windows needs a pre-built binary. The easiest way:

pip install pipwin
pipwin install pyaudio

Or download the correct .whl from https://www.lfd.uci.edu/~gohlke/pythonlibs/#pyaudio and install with:

pip install PyAudio‑0.2.14‑cpXX‑cpXX‑win_amd64.whl

4. Install remaining dependencies

pip install -r requirements.txt

5. Set your Gemini API key

Option A — .env file (recommended):

copy .env.example .env
# Then open .env and replace "your_gemini_api_key_here" with your actual key

Option B — edit config.py directly:

Open config.py and change:

GEMINI_API_KEY: str = os.getenv("GEMINI_API_KEY", "YOUR_GEMINI_API_KEY_HERE")

to:

GEMINI_API_KEY: str = "your_actual_api_key"

---

Running

Voice mode (default — microphone + TTS)

python main.py

Text-only mode (no microphone needed)

python main.py --text

List available TTS voices

python main.py --list-voices

---

Example Interactions

You say	What happens
"What's the weather in Tokyo?"	Calls get_weather MCP tool → speaks result
"Calculate 2 to the power of 32"	Calls calculator tool → speaks 4294967296
"Search for the latest Python news"	Calls web_search → summarises top results
"What day is today?"	Calls get_datetime → speaks date & time
"Read the file notes.txt"	Calls read_file → speaks file contents
"Write 'Hello World' to test.txt"	Calls write_file → creates/updates file
"Reset conversation"	Clears chat history
"Goodbye" / "Exit"	Exits the assistant

---

Project Structure

AI voice assistant/
├── main.py          # Entry point — CLI, banner, main loop
├── assistant.py     # Gemini + MCP integration (agentic tool-call loop)
├── speech.py        # SpeechRecognition (STT) + pyttsx3 (TTS)
├── mcp_server.py    # MCP tool server with 7 built-in tools
├── config.py        # All settings and API key placeholder
├── requirements.txt # Python dependencies
├── .env.example     # API key template
└── README.md        # This file

---

Customisation

Change the AI's personality

Edit SYSTEM_PROMPT in config.py.

Adjust microphone sensitivity

Edit MIC_ENERGY_THRESHOLD in config.py (lower = more sensitive).

Change TTS voice or speed

Edit TTS_RATE and TTS_VOICE_PREFERENCE in config.py. Run python main.py --list-voices to see available voice names.

Add more MCP tools

Open mcp_server.py, add a new function, then register it in list_tools() and call_tool().

---

Gemini Free Tier Limits

Limit	Value
Requests per minute	15
Tokens per day	1,000,000
Cost	$0

Get your key at: https://aistudio.google.com/app/apikey

---

Troubleshooting

"No module named 'pyaudio'" → See PyAudio installation step above.

"Could not understand audio" → Speak clearly; adjust MIC_ENERGY_THRESHOLD lower in config.py.

"Speech recognition service error" → Check your internet connection (Google Web Speech API requires internet).

Gemini 429 / rate limit error → You've hit the free tier limit. Wait a minute and try again.

Assistant doesn't speak / TTS silent → Check system audio / volume. Try python main.py --list-voices to verify pyttsx3 works.

---

License

MIT — free to use, modify, and distribute.