Discover Awesome MCP Servers

crawl4ai-mcp

Okay, here's a Python outline and conceptual structure for wrapping the Crawl4AI library within an MCP (Model Context Protocol) server. This is a complex task, so I'll break it down into key components and provide code snippets to illustrate the core ideas. **Conceptual Overview** 1. **Crawl4AI Library:** Assume you have the Crawl4AI library installed and accessible in your Python environment. This library provides functions for web crawling, data extraction, and AI-related tasks. Let's say it has functions like: * `crawl_website(url, max_depth)`: Crawls a website up to a specified depth. * `extract_text(html_content)`: Extracts text from HTML content. * `analyze_content(text)`: Analyzes the extracted text using AI models (e.g., sentiment analysis, topic extraction). 2. **MCP Server:** The MCP server will act as a central point for receiving requests to use the Crawl4AI functions. It will expose these functions as services that can be called remotely. We'll use a framework like Flask or FastAPI to create the server. 3. **MCP Protocol:** MCP defines a standard way for clients to communicate with the server. Requests are typically sent as JSON payloads, and responses are also in JSON format. The requests will specify which Crawl4AI function to call and the parameters to pass to it. 4. **Python Implementation:** We'll use Python to: * Wrap the Crawl4AI functions. * Create the MCP server using Flask or FastAPI. * Handle incoming MCP requests. * Call the Crawl4AI functions. * Format the results as JSON responses. **Code Structure (Illustrative)** ```python # Import necessary libraries from flask import Flask, request, jsonify import json # Assume Crawl4AI is installed and accessible # import Crawl4AI # Replace with the actual import app = Flask(__name__) # Mock Crawl4AI functions (replace with actual Crawl4AI calls) def crawl_website(url, max_depth): """Mocks crawling a website.""" print(f"Crawling {url} with max depth {max_depth}") # Simulate crawling and getting HTML content html_content = f"<html><body><h1>Crawled Content from {url}</h1><p>Some text.</p></body></html>" return html_content def extract_text(html_content): """Mocks extracting text from HTML.""" print("Extracting text from HTML") text = "Crawled Content from a website. Some text." return text def analyze_content(text): """Mocks analyzing content.""" print("Analyzing content") analysis_result = {"sentiment": "neutral", "topic": "general"} return analysis_result @app.route('/mcp', methods=['POST']) def mcp_handler(): """Handles MCP requests.""" try: data = request.get_json() print(f"Received MCP request: {data}") # Extract function name and parameters from the MCP request function_name = data.get('function') params = data.get('params', {}) # Default to empty dictionary if no params # Call the appropriate Crawl4AI function based on the function name if function_name == 'crawl_website': url = params.get('url') max_depth = params.get('max_depth', 1) # Default max_depth result = crawl_website(url, max_depth) # Call the Crawl4AI function response_data = {"result": result} # Wrap the result elif function_name == 'extract_text': html_content = params.get('html_content') result = extract_text(html_content) response_data = {"result": result} elif function_name == 'analyze_content': text = params.get('text') result = analyze_content(text) response_data = {"result": result} else: return jsonify({"error": "Invalid function name"}), 400 # Return the result as a JSON response return jsonify(response_data), 200 except Exception as e: print(f"Error processing request: {e}") return jsonify({"error": str(e)}), 500 if __name__ == '__main__': app.run(debug=True, host='0.0.0.0', port=5000) ``` **Explanation:** 1. **Imports:** Imports Flask for creating the web server and `json` for handling JSON data. 2. **Mock Crawl4AI Functions:** These are placeholder functions. **You must replace these with actual calls to your Crawl4AI library.** They simulate the behavior of the Crawl4AI functions for demonstration purposes. 3. **`mcp_handler` Function:** * This function is the endpoint that receives MCP requests (at the `/mcp` route). * It parses the JSON request body using `request.get_json()`. * It extracts the `function` name and `params` from the request. * It uses an `if/elif/else` block to determine which Crawl4AI function to call based on the `function_name`. * It calls the appropriate Crawl4AI function with the provided parameters. * It formats the result as a JSON response using `jsonify()`. * It handles potential errors using a `try...except` block and returns an error response if something goes wrong. 4. **`if __name__ == '__main__':`:** This ensures that the Flask app is only run when the script is executed directly (not when it's imported as a module). It starts the Flask development server. **How to Run:** 1. **Install Flask:** `pip install flask` 2. **Replace Mock Functions:** Replace the mock Crawl4AI functions with actual calls to your Crawl4AI library. 3. **Run the Script:** `python your_script_name.py` **Example MCP Request (sent to the server):** ```json { "function": "crawl_website", "params": { "url": "https://www.example.com", "max_depth": 2 } } ``` **Example MCP Response (from the server):** ```json { "result": "<html><body><h1>Crawled Content from https://www.example.com</h1><p>Some text.</p></body></html>" } ``` **Key Improvements and Considerations:** * **Error Handling:** The `try...except` block provides basic error handling. You should add more robust error handling, including logging and more specific exception handling. * **Input Validation:** Validate the input parameters in the `mcp_handler` function to prevent errors and security vulnerabilities. For example, check if the `url` is a valid URL. * **Security:** If this server will be exposed to the internet, implement proper security measures, such as authentication and authorization. Consider using HTTPS. * **Asynchronous Operations:** Web crawling can be time-consuming. Consider using asynchronous tasks (e.g., with `asyncio` or Celery) to prevent the server from blocking while crawling. This will improve the server's responsiveness. * **Configuration:** Use a configuration file (e.g., a `.ini` or `.yaml` file) to store settings such as the server port, logging level, and API keys. * **Logging:** Implement comprehensive logging to track requests, errors, and other important events. * **Documentation:** Document the API endpoints and the expected request/response formats. Consider using a tool like Swagger/OpenAPI to generate API documentation. * **Rate Limiting:** Implement rate limiting to prevent abuse of the API. * **Framework Choice:** While Flask is a good starting point, FastAPI is often preferred for modern APIs due to its performance and automatic data validation. The code structure would be similar, but you'd use FastAPI's decorators and data validation features. **Example using FastAPI:** ```python from fastapi import FastAPI, HTTPException from pydantic import BaseModel from typing import Optional app = FastAPI() # Define data models for request parameters (using Pydantic) class CrawlWebsiteParams(BaseModel): url: str max_depth: Optional[int] = 1 class ExtractTextParams(BaseModel): html_content: str class AnalyzeContentParams(BaseModel): text: str # Mock Crawl4AI functions (replace with actual Crawl4AI calls) def crawl_website(url: str, max_depth: int): """Mocks crawling a website.""" print(f"Crawling {url} with max depth {max_depth}") # Simulate crawling and getting HTML content html_content = f"<html><body><h1>Crawled Content from {url}</h1><p>Some text.</p></body></html>" return html_content def extract_text(html_content: str): """Mocks extracting text from HTML.""" print("Extracting text from HTML") text = "Crawled Content from a website. Some text." return text def analyze_content(text: str): """Mocks analyzing content.""" print("Analyzing content") analysis_result = {"sentiment": "neutral", "topic": "general"} return analysis_result @app.post("/crawl_website") async def crawl_website_endpoint(params: CrawlWebsiteParams): try: result = crawl_website(params.url, params.max_depth) return {"result": result} except Exception as e: raise HTTPException(status_code=500, detail=str(e)) @app.post("/extract_text") async def extract_text_endpoint(params: ExtractTextParams): try: result = extract_text(params.html_content) return {"result": result} except Exception as e: raise HTTPException(status_code=500, detail=str(e)) @app.post("/analyze_content") async def analyze_content_endpoint(params: AnalyzeContentParams): try: result = analyze_content(params.text) return {"result": result} except Exception as e: raise HTTPException(status_code=500, detail=str(e)) ``` In the FastAPI example: * **Pydantic:** Pydantic is used to define data models (`CrawlWebsiteParams`, `ExtractTextParams`, `AnalyzeContentParams`). This provides automatic data validation and serialization. * **Type Hints:** Type hints are used extensively to improve code readability and help with static analysis. * **FastAPI Decorators:** FastAPI's decorators (`@app.post`) are used to define the API endpoints and the HTTP methods they handle. * **HTTPException:** `HTTPException` is used to raise HTTP errors with appropriate status codes and error messages. * **Separate Endpoints:** Each Crawl4AI function has its own dedicated endpoint (e.g., `/crawl_website`, `/extract_text`). This is generally a cleaner and more RESTful approach than a single `/mcp` endpoint. Remember to replace the mock functions with your actual Crawl4AI library calls. This comprehensive outline should give you a solid foundation for building your MCP server. Good luck!

image_studio_mcp

An MCP server that lets any MCP client generate and edit images using Image Studio API, returning results inline and saving PNGs to disk.

Xident MCP

MCP servers for building and running Xident age and identity verification from AI agents. They provide documentation lookup, sandbox test verifications, webhook debugging, and a production runtime with OAuth 2.1 authorization.

InfluxDB-v1-MCP

InfluxDB-v1-MCP is a powerful Model Context Protocol (MCP) interface specifically designed for InfluxDB v1.x, enabling AI assistants to intelligently manage and query time-series databases.

ApexVol MCP Server

Provides natural language access to ApexVol's options analytics platform, enabling queries on options chains, volatility metrics, Greeks, flow analysis, and strategy building via Claude Code or Claude Desktop.

BharatPay MCP

Provides AI agents with Indian fintech utilities including IFSC bank lookups, PAN/GSTIN validation, mutual fund NAVs, UPI VPA identification, pincode lookups, and INR formatting, all with zero authentication and zero cost.

bubble-agent-mcp

MCP server that helps users discover fast-growing internet conversations and trending topics in real time. Compatible with AI assistants like Claude, Codex, and Cursor, it enables natural language search for emerging discussions.

Kuzu Knowledge Graph MCP Server

Connects to an embedded Kuzu graph database and provides read-only MCP tools for AI clients to query a knowledge graph.

wyt-mcp

Game master MCP server for a time-loop dungeon RPG, managing game state, dice rolls, and narrative for a wizard-trapped town.

knowmind

Das Agentengehirn und Langzeitgedächtnis aus Deutschland.

mcp-jsonplaceholder

Wraps the JSONPlaceholder fake REST API to enable testing and prototyping with CRUD operations on posts, comments, and other resources via MCP.

precision-desktop

MCP server that fixes Windows DPI coordinate scaling for AI desktop automation. Provides UI element finding via Windows Accessibility APIs and coordinate conversion between physical and logical coordinate systems.

Levels of Self

Provides access to thousands of interactive self-awareness scenarios, behavioral archetypes, and breakthrough exercises from the Levels of Self development game. It enables AI assistants to guide users through pattern recognition and developmental coaching across seven levels of self-awareness.

YAZIO MCP Server

MCP server over the YAZIO food diary that lets a model read what you've eaten today, log meals, build recipes from tracked products, track water, weight and exercise, and query nutrition summaries, goals and profiles.

python-sandbox-mcp-server

This is a secure Python code execution server that enables LLMs to run Python code safely in isolated Docker containers, supporting stdout capture and Matplotlib PNG generation.

tokenguard-mcp

MCP server providing 10 crypto/DeFi data tools (prices, gas, Fear & Greed, TVL, etc.) with no API keys required.

netbox-mcp

A read-only MCP server that exposes NetBox IPAM data over HTTP, allowing engineers to query subnets, IPs, and VLANs using natural language through AI assistants.

Strata

Local-first memory layer for AI agents. Compresses context with a local LLM and serves hybrid search over MCP.

Google Workspace MCP Server

Google Workspace MCP Server

Social MCP Server

Enables AI assistants to network on behalf of users, handling profile management, intent-based matching, and secure messaging through the Model Context Protocol.

LightWorker

Local-first multi-agent delegation and approval control for Codex via MCP, with persistent task DAG, isolated worktrees, and a web console.

MCP Kling

An integration that enables Claude to generate AI videos from text prompts or images using Kling AI's video generation models.

TestOps MCP Server

Enables AI assistants to interact with TestOps 5.25 for managing projects, test runs, test plans, and test cases via MCP tools.

Bitpanda Developer API MCP Server

Enables interaction with the Bitpanda cryptocurrency exchange through their Developer API. Exposes Bitpanda trading and account management endpoints as MCP tools via a FastAPI wrapper.

Databricks MCP Server

A Model Context Protocol server that provides tools for querying, updating, and exploring Databricks SQL Analytics tables.

Tianshang301/TianshangCAD

Modern CAD CLI + MCP Server: 2D/3D drawing, editing, measurement, validation, batch automation, real-time collaboration (CRDT + WebSocket).

fal

Enables MCP clients to run 600+ generative AI models from fal.ai, including image, video, audio, and text, with tools for synchronous and asynchronous execution, model catalog browsing, and schema inspection.

MCP Data Analyzer

Enables loading and statistical analysis of .xlsx and .csv files with visualization capabilities using matplotlib and plotly to generate various graphs and charts.

Etsy MCP Server

An MCP server for interacting with the Etsy API, enabling listing management, shop information, shipping profiles, and image uploads.

meetup-mcp-server

An MCP server for automating Meetup event tracking with an AI agent. It equips the agent with tools to browse Meetup.com, detect new events since the last known record, and return structured event data.