AI Peer Review MCP Server

Enhance your local LLM responses with real-time peer review from Google Gemini

A Model Context Protocol (MCP) server that enables local language models to request peer review feedback from Google Gemini, dramatically improving response quality through AI collaboration.

🌟 Features

• Real-time peer review from Google Gemini for any local LLM response
• Manual trigger system - user controls when to request peer review
• Detailed feedback analysis - accuracy, completeness, clarity, and improvement suggestions
• Comprehensive logging - see exactly what feedback Gemini provides
• Privacy-conscious - only shares content when explicitly requested
• Free to use - leverages Google Gemini's free tier
• Easy integration - works with any MCP-compatible local LLM setup

🎯 Use Cases

• Fact-checking complex or technical responses
• Quality improvement for educational content
• Writing enhancement for creative tasks
• Technical validation for coding explanations
• Research assistance with multiple AI perspectives

📋 Prerequisites

• Python 3.8+ installed on your system
• LMStudio (or another MCP-compatible LLM client)
• Google AI Studio account (free) for Gemini API access
• Local LLM with tool calling support (e.g., Llama 3.1, Mistral, Qwen)

🚀 Quick Start

1. Get Google Gemini API Key

1. Visit Google AI Studio
2. Sign in with your Google account
3. Click "Get API key" → "Create API key in new project"
4. Copy your API key (starts with AIza...)

2. Install the MCP Server

# Clone or create project directory
git clone https://github.com/your-repo/ai-peer-review-mcp # Replace with the actual repo URL
cd ai-peer-review-mcp

# Create a virtual environment (recommended)
python3 -m venv venv
source venv/bin/activate  # On Windows use `venv\Scripts\activate`

# Install dependencies
pip install -r requirements.txt

# Create environment file
cp .env.example .env
# Now, edit the .env file and add your API key:
# GEMINI_API_KEY=your_actual_api_key_here

3. Review Server Files

requirements.txt:

requests
python-dotenv

server.py: (See full code in the repository)

4. Configure LMStudio or any other supported MCP Host (e.g Claude Desktop)

Add this configuration to your LMStudio MCP settings:

{
  "mcpServers": {
    "ai-peer-review": {
      "command": "python",
      "args": ["server.py"],
      "cwd": "/path/to/your/ai-peer-review-mcp",
      "env": {
        "GEMINI_API_KEY": "your_actual_api_key_here"
      }
    }
  }
}

Finding MCP Settings in LMStudio:

• Look for: Settings → MCP Servers
• Or: Tools & Integrations → MCP Configuration
• Or: Program button → Edit MCP JSON

5. Test the Setup

1. Restart LMStudio after adding the MCP configuration
2. Start a new chat in LMStudio
3. Ask any question: "What is quantum computing?"
4. Request peer review: "Use the ai_peer_review tool to check and improve your answer"

📚 Usage Examples

Basic Usage

User: What causes climate change?

LLM: [Provides initial response about greenhouse gases...]

User: Use AI Peer Review to verify and improve that answer

LLM: [Calls ai_peer_review tool, receives feedback, provides enhanced response]

Technical Questions

User: Explain how neural networks work

LLM: [Initial technical explanation...]

User: Can you use ai_peer_review to make sure the explanation is accurate?

LLM: [Enhanced response with better technical details and examples]

Creative Tasks

User: Write a short story about AI

LLM: [Initial creative writing...]

User: Use peer review to improve the story structure and clarity

LLM: [Improved story with better narrative flow and character development]

🔧 Configuration Options

Environment Variables

• GEMINI_API_KEY - Your Google Gemini API key (required)

Customization

You can modify the peer review prompt in server.py to focus on specific aspects:

review_prompt = f"""PEER REVIEW REQUEST:
# Customize this section for your specific needs
# Examples:
# - Focus on technical accuracy for coding questions
# - Emphasize creativity for writing tasks
# - Prioritize safety for medical/legal topics
...
"""

📊 Monitoring and Logs

The server creates detailed logs in mcp-server.log:

# Watch logs in real-time
tail -f mcp-server.log

# View recent activity
cat mcp-server.log | tail -50

Log Information Includes:

• Tool calls from LMStudio
• Requests sent to Gemini
• Raw Gemini responses
• Parsed feedback
• Error details

🐛 Troubleshooting

Common Issues

"Tool not available"

• Verify MCP server configuration in LMStudio
• Ensure your local model supports tool calling
• Restart LMStudio after configuration changes

"GEMINI_API_KEY not found"

• Check your .env file exists and has the correct key
• Verify API key is valid in Google AI Studio
• Ensure environment variable is properly set in LMStudio config

"Rate limit exceeded"

• Google Gemini free tier has generous limits
• Wait a moment and try again
• Check Google AI Studio quota usage

"Model not found"

• API model names change over time
• Update GEMINI_API_URL in server.js if needed
• Check Google's latest API documentation

Debug Mode

Run the server manually to see detailed output. Make sure your virtual environment is active.

export GEMINI_API_KEY=your_api_key_here
python server.py

🔒 Privacy and Security

• Data sharing only on request - content is only sent to Gemini when explicitly triggered
• No persistent storage - conversations are not stored or logged beyond current session
• API key security - keep your Gemini API key private and secure
• Local processing - MCP runs entirely on your machine

🚧 Limitations

• Requires tool-calling models - basic instruction-following models won't work
• Internet connection required - needs access to Google Gemini API
• Rate limits - subject to Google Gemini API quotas (free tier is generous)
• Language support - optimized for English, other languages may work but aren't tested

🛣️ Roadmap

• [ ] Multi-provider support - Add Groq, DeepSeek, and other AI APIs
• [ ] Smart routing - Automatic provider selection based on question type
• [ ] Confidence thresholds - Auto-trigger peer review for uncertain responses
• [ ] Custom review templates - Domain-specific review criteria
• [ ] Usage analytics - Track improvement metrics and API usage
• [ ] Batch processing - Review multiple responses at once

🤝 Contributing

We welcome contributions! Here's how to help:

Development Setup

git clone https://github.com/your-repo/ai-peer-review-mcp # Replace with your repo URL
cd ai-peer-review-mcp

# Create and activate virtual environment
python3 -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Set up your environment
cp .env.example .env
# --> Add your GEMINI_API_KEY to the .env file

echo "Development environment ready. Run with 'python server.py'"

Ways to Contribute

• 🐛 Bug reports - Open issues for any problems you encounter
• 💡 Feature requests - Suggest new capabilities or improvements
• 📖 Documentation - Improve setup guides, add examples
• 🔧 Code contributions - Submit pull requests for fixes or features
• 🧪 Testing - Try with different models and report compatibility
• 🌍 Localization - Help support more languages

Contribution Guidelines

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Make your changes with clear, descriptive commits
4. Add tests if applicable
5. Update documentation for any new features
6. Submit a pull request with a clear description

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• Anthropic - For creating the Model Context Protocol standard
• Google - For providing the Gemini API
• LMStudio - For excellent MCP integration
• Community contributors - Everyone who helps improve this project

📞 Support

• Issues: GitHub Issues
• Discussions: GitHub Discussions

🌟 Star History

If this project helps you, please consider giving it a star on GitHub! ⭐

---

Made with ❤️ for the AI community