Discover Awesome MCP Servers

Valkey MCP Task Management Server

Enables AI agents to create, manage, and track tasks within plans using Valkey as the persistence layer. Supports plan and task management with Markdown notes, status tracking, and prioritization through multiple transport protocols (SSE, Streamable HTTP, STDIO).

Kedro RAG MCP

An MCP server that enables users to query Kedro framework documentation using retrieval-augmented generation. It builds a local knowledge base from documentation files to help users navigate and apply Kedro's data science pipeline framework.

Pagos Data MCP Server

Enables Claude to retrieve BIN (Bank Identification Number) data for payment cards, with options for basic or enhanced insights through the Pagos API.

Persistent Shell MCP

Enables AI assistants to execute shell commands and manage long-running processes within persistent tmux sessions across isolated workspaces. It features a dual-window architecture to separate raw command execution from interactive terminal output.

ZBD MCP Server

A server that adds Bitcoin payment capabilities to LLMs, enabling sending/receiving payments, creating charges, managing wallets, and performing other Bitcoin Lightning Network operations.

Stellar MCP

Enables interaction with Stellar Classic and Soroban smart contracts, allowing users to manage accounts, process payments, and handle asset operations. It provides comprehensive tools for building, deploying, and retrieving the interfaces of smart contracts on the Stellar network.

PortOne Global MCP Server

Enables searching and reading of PortOne documentation, including OpenAPI schemas and product guides, through the Model Context Protocol. It allows AI agents to easily access and integrate payment-related technical specifications into their workflows.

MCP Server Markup Language (MCPML)

MCP Server Markup Language (MCPML) - CLIとOpenAIエージェントのサポートを備えたMCPサーバーを構築するためのPythonフレームワークです。

Enrichment MCP Server

A Model Context Protocol server that enables users to perform third-party enrichment lookups for security observables (IP addresses, domains, URLs, emails) through services like VirusTotal, Shodan, and others.

SQLx MCP Server

Provides comprehensive database management tools for PostgreSQL, MySQL, and SQLite databases. Enables querying table structures, executing read-only and write queries, exporting DDL statements, and managing database metadata through natural language.

MCP Google Calendar Server

Enables creating and managing Google Calendar events through OAuth 2.0 authentication, supporting event details like title, time, location, description, and attendees.

MCP server for LogSeq

Interacts with LogSeq via its API.

Claude Kali MCP Commander

An MCP server that provides secure access to Kali Linux cybersecurity tools through Claude's interface, enabling users to run Kali Linux commands directly from Claude Desktop.

Echo MCP Server

MCP (Model Context Protocol) クライアントのテスト用に設計された、メッセージをそのままエコーバックするシンプルな MCP サーバー。

Fetch MCP Server

Okay, here's a breakdown of how you can fetch URLs from a webpage using Playwright, integrate it with an SSE (Server-Sent Events) MCP (Management Control Protocol) server, and use Node.js with Express.js to manage the whole process. I'll provide code snippets and explanations to guide you. **Conceptual Overview** 1. **Playwright (Web Scraping):** Playwright will be used to launch a browser, navigate to the target webpage, and extract the URLs you need. 2. **Node.js with Express.js (Server):** Express.js will create a web server that handles requests to start the scraping process and stream the results back to the client using Server-Sent Events (SSE). 3. **SSE (Server-Sent Events):** SSE is a one-way communication protocol where the server pushes updates to the client in real-time. This is ideal for streaming the URLs as they are discovered. 4. **MCP (Management Control Protocol):** The MCP part is a bit vague without more context. I'll assume it means you want to control and monitor the scraping process. I'll include basic mechanisms for starting/stopping the scraping and potentially reporting status. If you have specific MCP requirements (e.g., a particular protocol or data format), please provide more details. **Code Structure** I'll break the code into three main parts: * **`server.js` (Node.js/Express.js Server):** Handles requests, starts Playwright, and streams URLs via SSE. * **`playwright_scraper.js` (Playwright Script):** The core logic for scraping URLs from a webpage. * **`client.html` (Simple Client):** A basic HTML page to connect to the SSE stream and display the URLs. **1. `server.js` (Node.js/Express.js Server)** ```javascript const express = require('express'); const { spawn } = require('child_process'); // For running Playwright script const cors = require('cors'); // Import the cors middleware const app = express(); const port = 3000; app.use(cors()); // Enable CORS for all routes let scraperProcess = null; // Store the Playwright process let sseClients = []; // Array to hold connected SSE clients // SSE setup app.get('/events', (req, res) => { res.setHeader('Content-Type', 'text/event-stream'); res.setHeader('Cache-Control', 'no-cache'); res.setHeader('Connection', 'keep-alive'); res.flushHeaders(); const clientId = Date.now(); sseClients.push({ id: clientId, res }); console.log(`Client ${clientId} connected`); // Send a ping every 30 seconds to keep the connection alive const pingInterval = setInterval(() => { sseClients.find(client => client.id === clientId)?.res.write(`: ping\n\n`); }, 30000); req.on('close', () => { console.log(`Client ${clientId} disconnected`); sseClients = sseClients.filter(client => client.id !== clientId); clearInterval(pingInterval); }); }); function sendSSE(data) { sseClients.forEach(client => { client.res.write(`data: ${JSON.stringify(data)}\n\n`); }); } // Start scraping endpoint app.post('/start-scraping', (req, res) => { if (scraperProcess) { return res.status(400).send('Scraping already in progress.'); } scraperProcess = spawn('node', ['playwright_scraper.js']); scraperProcess.stdout.on('data', (data) => { const message = data.toString().trim(); try { const parsedData = JSON.parse(message); sendSSE(parsedData); } catch (error) { console.error("Error parsing JSON:", error); console.log("Received data:", message); // Log the raw message sendSSE({ type: 'log', message: message }); // Send as a log message } }); scraperProcess.stderr.on('data', (data) => { console.error(`Scraper error: ${data}`); sendSSE({ type: 'error', message: data.toString() }); }); scraperProcess.on('close', (code) => { console.log(`Scraper process exited with code ${code}`); scraperProcess = null; sendSSE({ type: 'status', message: `Scraping finished with code ${code}` }); }); res.status(202).send('Scraping started.'); // Accepted }); // Stop scraping endpoint app.post('/stop-scraping', (req, res) => { if (!scraperProcess) { return res.status(400).send('Scraping not in progress.'); } scraperProcess.kill('SIGINT'); // Or 'SIGTERM' scraperProcess = null; res.send('Scraping stopped.'); }); app.listen(port, () => { console.log(`Server listening at http://localhost:${port}`); }); ``` **Explanation of `server.js`:** * **Dependencies:** Requires `express`, `child_process`, and `cors`. Install them: `npm install express child_process cors` * **CORS:** `app.use(cors());` enables Cross-Origin Resource Sharing, allowing your client (e.g., a webpage on a different domain) to access the server. Important for web applications. * **`scraperProcess`:** Stores a reference to the Playwright process. This is crucial for stopping the process later. * **`/events` (SSE Endpoint):** * Sets the correct headers for SSE. * Creates a new client object and adds it to the `sseClients` array. * `req.on('close')`: Handles client disconnections, removing the client from the array and clearing the ping interval. * `sendSSE(data)`: Sends data to all connected SSE clients. It stringifies the data as JSON. * **Ping:** Sends a ping every 30 seconds to keep the connection alive. Some browsers or proxies might close idle SSE connections. * **`/start-scraping` (POST):** * Checks if scraping is already in progress. * Uses `child_process.spawn` to run the `playwright_scraper.js` script. This runs the script in a separate process. * **`stdout.on('data')`:** Captures the output from the Playwright script (URLs, logs, etc.). It parses the output as JSON and sends it to the SSE clients. Includes error handling for JSON parsing. * **`stderr.on('data')`:** Captures any errors from the Playwright script and sends them to the SSE clients. * **`on('close')`:** Handles the Playwright process exiting. Clears the `scraperProcess` variable and sends a status message to the clients. * Returns a 202 Accepted status code to indicate that the scraping process has been started. * **`/stop-scraping` (POST):** * Checks if scraping is in progress. * Uses `scraperProcess.kill('SIGINT')` to stop the Playwright process. `SIGINT` is a signal that tells the process to terminate gracefully. You can also use `SIGTERM`. * **Error Handling:** The `stdout.on('data')` includes a `try...catch` block to handle potential errors when parsing the JSON output from the Playwright script. This is important because the script might output log messages or other non-JSON data. * **Logging:** The code includes `console.log` statements to help you debug the server. **2. `playwright_scraper.js` (Playwright Script)** ```javascript const { chromium } = require('playwright'); async function scrapeUrls(url) { const browser = await chromium.launch({ headless: true }); // Set headless to false to see the browser const page = await browser.newPage(); try { await page.goto(url); // Wait for the page to load completely (adjust the timeout if needed) await page.waitForLoadState('networkidle', { timeout: 60000 }); // Extract all links (href attributes) const links = await page.evaluate(() => { const anchors = Array.from(document.querySelectorAll('a')); return anchors.map(anchor => anchor.href).filter(href => href.startsWith('http')); // Filter out relative links }); // Remove duplicate links const uniqueLinks = [...new Set(links)]; // Send each unique link to the server for (const link of uniqueLinks) { console.log(JSON.stringify({ type: 'url', url: link })); // Output as JSON } console.log(JSON.stringify({ type: 'status', message: 'Scraping completed successfully' })); } catch (error) { console.error("Scraping failed:", error); console.log(JSON.stringify({ type: 'error', message: error.message })); // Send error to server } finally { await browser.close(); } } // Get the target URL from environment variables or use a default const targetUrl = process.env.TARGET_URL || 'https://www.example.com'; // Replace with your target URL scrapeUrls(targetUrl); ``` **Explanation of `playwright_scraper.js`:** * **Dependencies:** Requires `playwright`. Install it: `npm install playwright` * **`scrapeUrls(url)`:** The main scraping function. * Launches a Chromium browser in headless mode (no GUI). Set `headless: false` to see the browser window. * Creates a new page. * Navigates to the specified URL using `page.goto(url)`. * **`page.waitForLoadState('networkidle')`:** Waits for the page to load completely. `networkidle` means that there are no network connections for at least 500ms. Adjust the `timeout` if needed. * **`page.evaluate()`:** Executes JavaScript code in the context of the browser page. This is how you access the DOM. * It selects all `<a>` elements. * It extracts the `href` attribute of each link. * It filters out relative links (links that don't start with `http`). * **`new Set(links)`:** Removes duplicate links. * **`console.log(JSON.stringify({ type: 'url', url: link }))`:** This is the crucial part for SSE. It outputs each URL to the console as a JSON string. The server will capture this output and send it to the client. The `type` field is used to identify the type of data being sent (URL, status, error, etc.). * **Error Handling:** The `try...catch` block handles any errors that occur during the scraping process. It sends an error message to the server. * **`finally`:** Ensures that the browser is closed, even if an error occurs. * **`targetUrl`:** Gets the target URL from the `TARGET_URL` environment variable. If the environment variable is not set, it uses a default URL (`https://www.example.com`). This makes the script more configurable. To set the environment variable, you can run the server like this: `TARGET_URL=https://www.example.com node server.js` **3. `client.html` (Simple Client)** ```html <!DOCTYPE html> <html> <head> <title>SSE URL Stream</title> </head> <body> <h1>SSE URL Stream</h1> <ul id="urlList"></ul> <script> const urlList = document.getElementById('urlList'); const eventSource = new EventSource('http://localhost:3000/events'); // Replace with your server URL eventSource.onmessage = (event) => { const data = JSON.parse(event.data); if (data.type === 'url') { const listItem = document.createElement('li'); listItem.textContent = data.url; urlList.appendChild(listItem); } else if (data.type === 'status') { const listItem = document.createElement('li'); listItem.textContent = `Status: ${data.message}`; urlList.appendChild(listItem); } else if (data.type === 'error') { const listItem = document.createElement('li'); listItem.textContent = `Error: ${data.message}`; urlList.appendChild(listItem); } else if (data.type === 'log') { const listItem = document.createElement('li'); listItem.textContent = `Log: ${data.message}`; urlList.appendChild(listItem); } }; eventSource.onerror = (error) => { console.error("SSE error:", error); const listItem = document.createElement('li'); listItem.textContent = `SSE Error: ${error}`; urlList.appendChild(listItem); eventSource.close(); }; </script> <button id="startButton">Start Scraping</button> <button id="stopButton">Stop Scraping</button> <script> const startButton = document.getElementById('startButton'); const stopButton = document.getElementById('stopButton'); startButton.addEventListener('click', () => { fetch('http://localhost:3000/start-scraping', { method: 'POST' }) .then(response => { if (response.ok) { console.log('Scraping started'); } else { console.error('Failed to start scraping:', response.status); } }); }); stopButton.addEventListener('click', () => { fetch('http://localhost:3000/stop-scraping', { method: 'POST' }) .then(response => { if (response.ok) { console.log('Scraping stopped'); } else { console.error('Failed to stop scraping:', response.status); } }); }); </script> </body> </html> ``` **Explanation of `client.html`:** * **SSE Connection:** Creates an `EventSource` object to connect to the `/events` endpoint on the server. * **`eventSource.onmessage`:** Handles incoming SSE messages. * Parses the JSON data from the event. * Checks the `type` field to determine the type of data (URL, status, error). * Creates a new list item and adds it to the `urlList` element. * **`eventSource.onerror`:** Handles SSE errors. * **Start/Stop Buttons:** The HTML includes buttons to start and stop the scraping process. These buttons send POST requests to the `/start-scraping` and `/stop-scraping` endpoints on the server. * **Error Handling:** The client includes basic error handling for SSE connections and HTTP requests. **How to Run the Code** 1. **Install Node.js:** Make sure you have Node.js installed. 2. **Create a Project Directory:** Create a new directory for your project. 3. **Initialize the Project:** Run `npm init -y` in the project directory to create a `package.json` file. 4. **Install Dependencies:** Run `npm install express child_process playwright cors` 5. **Create Files:** Create the `server.js`, `playwright_scraper.js`, and `client.html` files in the project directory. 6. **Run the Server:** Run `node server.js` in the project directory. 7. **Open `client.html`:** Open the `client.html` file in your web browser. 8. **Click "Start Scraping":** Click the "Start Scraping" button to start the scraping process. 9. **See the URLs:** The URLs will be displayed in the list as they are discovered. 10. **Click "Stop Scraping":** Click the "Stop Scraping" button to stop the scraping process. **Important Considerations and Improvements** * **Error Handling:** Implement more robust error handling in both the server and the Playwright script. Log errors to a file or a monitoring service. * **Configuration:** Use environment variables or a configuration file to store settings such as the target URL, the scraping interval, and the log file path. * **Scalability:** For large-scale scraping, consider using a message queue (e.g., RabbitMQ or Kafka) to distribute the scraping tasks across multiple workers. * **Rate Limiting:** Implement rate limiting to avoid overloading the target website. Use `page.waitForTimeout()` to pause between requests. * **User Agent:** Set a realistic user agent to avoid being blocked by the target website. * **Proxies:** Use proxies to avoid being blocked by the target website. * **Data Storage:** Store the scraped data in a database or a file. * **MCP (Management Control Protocol):** If you have specific MCP requirements, you'll need to implement the appropriate protocol and data format. This might involve adding more endpoints to the server to handle MCP commands. * **Headless Mode:** Running in headless mode is generally recommended for performance. However, some websites may require a full browser to render correctly. You can set `headless: false` in the Playwright script to see the browser window. * **Resource Management:** Be mindful of resource usage, especially memory. Close the browser and pages when they are no longer needed. * **Legal and Ethical Considerations:** Always respect the target website's terms of service and robots.txt file. Avoid scraping data that you are not authorized to access. This comprehensive example provides a solid foundation for building a web scraping application with Playwright, Node.js, Express.js, and SSE. Remember to adapt the code to your specific needs and requirements. Good luck!

Spotify MCP Node Server

Enables AI assistants to control Spotify playback, manage playlists, search music, and access listening history. Requires Spotify Premium and uses secure OAuth 2.0 with PKCE authentication.

Yandex Search MCP Server

Enables AI assistants to perform real-time web searches and retrieve AI-generated answers using the Yandex Search API. It provides tools for accessing up-to-date internet information with support for both raw search results and summarized content via the Yazeka model.

YouTube Knowledge MCP

Transforms YouTube into a queryable knowledge source with search, video details, transcript analysis, and AI-powered tools for summaries, learning paths, and knowledge graphs. Features quota-aware API access with caching and optional OpenAI/Anthropic integration for advanced content analysis.

rest-to-mcp

REST API を MCP サーバーに変換するチュートリアルプロジェクト

chromium-arm64

MCP server that enables browser automation and web testing on ARM64 devices like Raspberry Pi, allowing users to navigate websites, take screenshots, execute JavaScript, and perform UI testing via Claude.

Weather MCP Server

A Model Context Protocol server that provides weather information and forecasts based on user location or address input.

mcd-demo

簡単な Minecraft サーバーの作成と、LangChain エージェントとの統合をテストしています。

Remote MCP Server

A serverless implementation of Model Context Protocol (MCP) on Cloudflare Workers that allows AI models to access custom tools without authentication.

MCP Synth Controller

Enables LLMs to control synthesizer parameters in real-time by translating natural language commands into OSC messages sent to a JUCE synthesizer application.

stackzero-labs/mcp

A model context protocol server that enables applications to use stackzero-labs/ui components through the MCP protocol, supporting both standalone operation and integration with Claude Desktop and Cursor.

ABS MCP Server

An MCP server that provides AI assistants with access to Australian Bureau of Statistics data through the SDMX-ML API, enabling statistical data querying and analysis.

ynab-mcp-server

鏡 (Kagami)

Senior Life Care MCP

Provides 20 tools across 6 categories (pension/welfare, health/medical, care, leisure/culture, living support, and safety/emergency) to help seniors aged 60+ easily understand and access Korean welfare services and benefits.

MCPDemo - Visual SQL Chat Platform

An AI-driven data visualization and SQL chat platform that enables natural language to SQL conversion, interactive chart generation, and database querying through a comprehensive MCP interface. Supports multiple databases, intelligent chart recommendations, and secure data management with temporary link sharing.