Discover Awesome MCP Servers
Extend your agent with 84,517 capabilities via MCP servers.
- All84,517
- Developer Tools3,867
- Search1,714
- Research & Data1,557
- AI Integration Systems229
- Cloud Platforms219
- Data & App Analysis181
- Database Interaction177
- Remote Shell Execution165
- Browser Automation147
- Databases145
- Communication137
- AI Content Generation127
- OS Automation120
- Programming Docs Access109
- Content Fetching108
- Note Taking97
- File Systems96
- Version Control93
- Finance91
- Knowledge & Memory90
- Monitoring79
- Security71
- Image & Video Processing69
- Digital Note Management66
- AI Memory Systems62
- Advanced AI Reasoning59
- Git Management Tools58
- Cloud Storage51
- Entertainment & Media43
- Virtualization42
- Location Services35
- Web Automation & Stealth32
- Media Content Processing32
- Calendar Management26
- Ecommerce & Retail18
- Speech Processing18
- Customer Data Platforms16
- Travel & Transportation14
- Education & Learning Tools13
- Home Automation & IoT13
- Web Search Integration12
- Health & Wellness10
- Customer Support10
- Marketing9
- Games & Gamification8
- Google Cloud Integrations7
- Art & Culture4
- Language Translation3
- Legal & Compliance2
Trino MCP Server
Enables database schema analysis and management for Trino servers through dynamic connections. Supports DDL validation, dependency analysis, schema documentation generation, and safe SQL execution with multiple concurrent connections.
qenex-pathfinder
Provides security scanning tools for Claude Code, enabling real-time vulnerability detection, code analysis, and rule explanations for codebases.
72bpm-leadgen-mcp-server
An MCP server that turns discovery → enrichment → scoring → outreach into callable tools, scoped to 72BPM's four practice areas.
Cost Explorer MCP
Estimates Claude API costs with interactive sliders, live comparison across five models, and returns the selected model to the conversation.
adclip
Generates ad creative (copy and images) from a structured JSON brief across Meta, Google, LinkedIn, and X formats. Includes self-review loops for policy compliance and variant scoring, and can run without API keys using Claude CLI.
Verificação de Identidade (Argentina)
Verifies a person's identity in Argentina by cross-checking provided data (DNI, name, birth, and address). Read-only MCP server with one tool, works with any MCP client over HTTP.
pa-dmv
Query Pennsylvania DMV data for EV adoption and vehicle registrations by county and ZIP code.
wiki-helper
Enables querying a Git-stored Markdown wiki by syncing, indexing, and providing tools to list, search, and read documents via MCP.
FireScrape MCP Server
Enables AI models to search the web using DuckDuckGo, scrape webpage content in markdown format, and browse/read local files for code analysis and debugging.
DAV MCP Server
A Model Context Protocol server that allows users to interact with their CalDAV, CardDAV, and WebDAV services (calendars, contacts, and files) from Fastmail and Apple iCloud accounts.
vision-mcp
Enables text-only LLMs to see images/videos via cloud vision models, offering vision chat, OCR, grounding, and media info tools with free GLM fallback.
Memex-Wiki MCP Server
Provides AI agents with a secure offline-first long-term memory system using Obsidian Markdown vault and 3D knowledge graph, with RLM-based querying to avoid context overload.
mcp-http
Provides a generic HTTP client tool allowing AI agents to make arbitrary HTTP requests from their environment, with support for environment-variable placeholders for secrets.
Lightbrush Booking Agent MCP
A development framework for restaurant booking automation that is currently only ~5% functional with mock implementations. The actual booking functionality, MCP integration, and web automation features are not yet implemented.
mcp-watchmode
Provides streaming availability data across 200+ services, including title search, releases, and title catalog.
GitLab MCP Server
mcp-permid
Provides access to Refinitiv/LSEG Permanent Identifiers (PermID) through an MCP gateway, enabling AI agents to query financial instrument and entity data via natural language or direct tool calls.
MonkeyMCP
This MCP server provides tools to retrieve monkey data and generate unique simulated journeys with species-specific activities and health statistics, enabling interaction with monkey-related information and simulations.
skillhub-mcp
Bridges Claude-style skills to MCP tools, enabling any MCP client to call the same skills for cross-client reuse and flexible packaging.
Stripe MCP Server
Enables natural language interaction with Stripe accounts to query customers, revenue, invoices, subscriptions, disputes, and issue refunds.
mcp-pionex
Enables AI assistants to interact with the Pionex exchange through MCP, providing market data, account info, spot trading, grid bots, and Dual Investment tools with strict anti-hallucination guardrails including two-phase order confirmation and live symbol/price validation.
CST MCP
An MCP server that integrates CST Studio Suite with the Model Context Protocol to enable automated electromagnetic simulation workflows. It provides specialized tools for material definition management and direct interaction with the CST Studio Suite environment.
OpenAPI MCP
A generic MCP server that builds its tools at runtime from any OpenAPI specification, enabling interaction with any REST API through natural language.
Tenjin MCP Server
Enables agents to discover, read, buy, and publish knowledge using USDC on Base, with search and free essays at no cost.
mcp-server-redmine
Enables AI assistants to create, update, and query Redmine issues and projects directly from conversations.
Fetch MCP Server
Okay, here's a breakdown of how you can fetch URLs from a webpage using Playwright, integrate it with an SSE (Server-Sent Events) MCP (Management Control Protocol) server, and use Node.js with Express.js to manage the whole process. I'll provide code snippets and explanations to guide you. **Conceptual Overview** 1. **Playwright (Web Scraping):** Playwright will be used to launch a browser, navigate to the target webpage, and extract the URLs you need. 2. **Node.js with Express.js (Server):** Express.js will create a web server that handles requests to start the scraping process and stream the results back to the client using Server-Sent Events (SSE). 3. **SSE (Server-Sent Events):** SSE is a one-way communication protocol where the server pushes updates to the client in real-time. This is ideal for streaming the URLs as they are discovered. 4. **MCP (Management Control Protocol):** The MCP part is a bit vague without more context. I'll assume it means you want to control and monitor the scraping process. I'll include basic mechanisms for starting/stopping the scraping and potentially reporting status. If you have specific MCP requirements (e.g., a particular protocol or data format), please provide more details. **Code Structure** I'll break the code into three main parts: * **`server.js` (Node.js/Express.js Server):** Handles requests, starts Playwright, and streams URLs via SSE. * **`playwright_scraper.js` (Playwright Script):** The core logic for scraping URLs from a webpage. * **`client.html` (Simple Client):** A basic HTML page to connect to the SSE stream and display the URLs. **1. `server.js` (Node.js/Express.js Server)** ```javascript const express = require('express'); const { spawn } = require('child_process'); // For running Playwright script const cors = require('cors'); // Import the cors middleware const app = express(); const port = 3000; app.use(cors()); // Enable CORS for all routes let scraperProcess = null; // Store the Playwright process let sseClients = []; // Array to hold connected SSE clients // SSE setup app.get('/events', (req, res) => { res.setHeader('Content-Type', 'text/event-stream'); res.setHeader('Cache-Control', 'no-cache'); res.setHeader('Connection', 'keep-alive'); res.flushHeaders(); const clientId = Date.now(); sseClients.push({ id: clientId, res }); console.log(`Client ${clientId} connected`); // Send a ping every 30 seconds to keep the connection alive const pingInterval = setInterval(() => { sseClients.find(client => client.id === clientId)?.res.write(`: ping\n\n`); }, 30000); req.on('close', () => { console.log(`Client ${clientId} disconnected`); sseClients = sseClients.filter(client => client.id !== clientId); clearInterval(pingInterval); }); }); function sendSSE(data) { sseClients.forEach(client => { client.res.write(`data: ${JSON.stringify(data)}\n\n`); }); } // Start scraping endpoint app.post('/start-scraping', (req, res) => { if (scraperProcess) { return res.status(400).send('Scraping already in progress.'); } scraperProcess = spawn('node', ['playwright_scraper.js']); scraperProcess.stdout.on('data', (data) => { const message = data.toString().trim(); try { const parsedData = JSON.parse(message); sendSSE(parsedData); } catch (error) { console.error("Error parsing JSON:", error); console.log("Received data:", message); // Log the raw message sendSSE({ type: 'log', message: message }); // Send as a log message } }); scraperProcess.stderr.on('data', (data) => { console.error(`Scraper error: ${data}`); sendSSE({ type: 'error', message: data.toString() }); }); scraperProcess.on('close', (code) => { console.log(`Scraper process exited with code ${code}`); scraperProcess = null; sendSSE({ type: 'status', message: `Scraping finished with code ${code}` }); }); res.status(202).send('Scraping started.'); // Accepted }); // Stop scraping endpoint app.post('/stop-scraping', (req, res) => { if (!scraperProcess) { return res.status(400).send('Scraping not in progress.'); } scraperProcess.kill('SIGINT'); // Or 'SIGTERM' scraperProcess = null; res.send('Scraping stopped.'); }); app.listen(port, () => { console.log(`Server listening at http://localhost:${port}`); }); ``` **Explanation of `server.js`:** * **Dependencies:** Requires `express`, `child_process`, and `cors`. Install them: `npm install express child_process cors` * **CORS:** `app.use(cors());` enables Cross-Origin Resource Sharing, allowing your client (e.g., a webpage on a different domain) to access the server. Important for web applications. * **`scraperProcess`:** Stores a reference to the Playwright process. This is crucial for stopping the process later. * **`/events` (SSE Endpoint):** * Sets the correct headers for SSE. * Creates a new client object and adds it to the `sseClients` array. * `req.on('close')`: Handles client disconnections, removing the client from the array and clearing the ping interval. * `sendSSE(data)`: Sends data to all connected SSE clients. It stringifies the data as JSON. * **Ping:** Sends a ping every 30 seconds to keep the connection alive. Some browsers or proxies might close idle SSE connections. * **`/start-scraping` (POST):** * Checks if scraping is already in progress. * Uses `child_process.spawn` to run the `playwright_scraper.js` script. This runs the script in a separate process. * **`stdout.on('data')`:** Captures the output from the Playwright script (URLs, logs, etc.). It parses the output as JSON and sends it to the SSE clients. Includes error handling for JSON parsing. * **`stderr.on('data')`:** Captures any errors from the Playwright script and sends them to the SSE clients. * **`on('close')`:** Handles the Playwright process exiting. Clears the `scraperProcess` variable and sends a status message to the clients. * Returns a 202 Accepted status code to indicate that the scraping process has been started. * **`/stop-scraping` (POST):** * Checks if scraping is in progress. * Uses `scraperProcess.kill('SIGINT')` to stop the Playwright process. `SIGINT` is a signal that tells the process to terminate gracefully. You can also use `SIGTERM`. * **Error Handling:** The `stdout.on('data')` includes a `try...catch` block to handle potential errors when parsing the JSON output from the Playwright script. This is important because the script might output log messages or other non-JSON data. * **Logging:** The code includes `console.log` statements to help you debug the server. **2. `playwright_scraper.js` (Playwright Script)** ```javascript const { chromium } = require('playwright'); async function scrapeUrls(url) { const browser = await chromium.launch({ headless: true }); // Set headless to false to see the browser const page = await browser.newPage(); try { await page.goto(url); // Wait for the page to load completely (adjust the timeout if needed) await page.waitForLoadState('networkidle', { timeout: 60000 }); // Extract all links (href attributes) const links = await page.evaluate(() => { const anchors = Array.from(document.querySelectorAll('a')); return anchors.map(anchor => anchor.href).filter(href => href.startsWith('http')); // Filter out relative links }); // Remove duplicate links const uniqueLinks = [...new Set(links)]; // Send each unique link to the server for (const link of uniqueLinks) { console.log(JSON.stringify({ type: 'url', url: link })); // Output as JSON } console.log(JSON.stringify({ type: 'status', message: 'Scraping completed successfully' })); } catch (error) { console.error("Scraping failed:", error); console.log(JSON.stringify({ type: 'error', message: error.message })); // Send error to server } finally { await browser.close(); } } // Get the target URL from environment variables or use a default const targetUrl = process.env.TARGET_URL || 'https://www.example.com'; // Replace with your target URL scrapeUrls(targetUrl); ``` **Explanation of `playwright_scraper.js`:** * **Dependencies:** Requires `playwright`. Install it: `npm install playwright` * **`scrapeUrls(url)`:** The main scraping function. * Launches a Chromium browser in headless mode (no GUI). Set `headless: false` to see the browser window. * Creates a new page. * Navigates to the specified URL using `page.goto(url)`. * **`page.waitForLoadState('networkidle')`:** Waits for the page to load completely. `networkidle` means that there are no network connections for at least 500ms. Adjust the `timeout` if needed. * **`page.evaluate()`:** Executes JavaScript code in the context of the browser page. This is how you access the DOM. * It selects all `<a>` elements. * It extracts the `href` attribute of each link. * It filters out relative links (links that don't start with `http`). * **`new Set(links)`:** Removes duplicate links. * **`console.log(JSON.stringify({ type: 'url', url: link }))`:** This is the crucial part for SSE. It outputs each URL to the console as a JSON string. The server will capture this output and send it to the client. The `type` field is used to identify the type of data being sent (URL, status, error, etc.). * **Error Handling:** The `try...catch` block handles any errors that occur during the scraping process. It sends an error message to the server. * **`finally`:** Ensures that the browser is closed, even if an error occurs. * **`targetUrl`:** Gets the target URL from the `TARGET_URL` environment variable. If the environment variable is not set, it uses a default URL (`https://www.example.com`). This makes the script more configurable. To set the environment variable, you can run the server like this: `TARGET_URL=https://www.example.com node server.js` **3. `client.html` (Simple Client)** ```html <!DOCTYPE html> <html> <head> <title>SSE URL Stream</title> </head> <body> <h1>SSE URL Stream</h1> <ul id="urlList"></ul> <script> const urlList = document.getElementById('urlList'); const eventSource = new EventSource('http://localhost:3000/events'); // Replace with your server URL eventSource.onmessage = (event) => { const data = JSON.parse(event.data); if (data.type === 'url') { const listItem = document.createElement('li'); listItem.textContent = data.url; urlList.appendChild(listItem); } else if (data.type === 'status') { const listItem = document.createElement('li'); listItem.textContent = `Status: ${data.message}`; urlList.appendChild(listItem); } else if (data.type === 'error') { const listItem = document.createElement('li'); listItem.textContent = `Error: ${data.message}`; urlList.appendChild(listItem); } else if (data.type === 'log') { const listItem = document.createElement('li'); listItem.textContent = `Log: ${data.message}`; urlList.appendChild(listItem); } }; eventSource.onerror = (error) => { console.error("SSE error:", error); const listItem = document.createElement('li'); listItem.textContent = `SSE Error: ${error}`; urlList.appendChild(listItem); eventSource.close(); }; </script> <button id="startButton">Start Scraping</button> <button id="stopButton">Stop Scraping</button> <script> const startButton = document.getElementById('startButton'); const stopButton = document.getElementById('stopButton'); startButton.addEventListener('click', () => { fetch('http://localhost:3000/start-scraping', { method: 'POST' }) .then(response => { if (response.ok) { console.log('Scraping started'); } else { console.error('Failed to start scraping:', response.status); } }); }); stopButton.addEventListener('click', () => { fetch('http://localhost:3000/stop-scraping', { method: 'POST' }) .then(response => { if (response.ok) { console.log('Scraping stopped'); } else { console.error('Failed to stop scraping:', response.status); } }); }); </script> </body> </html> ``` **Explanation of `client.html`:** * **SSE Connection:** Creates an `EventSource` object to connect to the `/events` endpoint on the server. * **`eventSource.onmessage`:** Handles incoming SSE messages. * Parses the JSON data from the event. * Checks the `type` field to determine the type of data (URL, status, error). * Creates a new list item and adds it to the `urlList` element. * **`eventSource.onerror`:** Handles SSE errors. * **Start/Stop Buttons:** The HTML includes buttons to start and stop the scraping process. These buttons send POST requests to the `/start-scraping` and `/stop-scraping` endpoints on the server. * **Error Handling:** The client includes basic error handling for SSE connections and HTTP requests. **How to Run the Code** 1. **Install Node.js:** Make sure you have Node.js installed. 2. **Create a Project Directory:** Create a new directory for your project. 3. **Initialize the Project:** Run `npm init -y` in the project directory to create a `package.json` file. 4. **Install Dependencies:** Run `npm install express child_process playwright cors` 5. **Create Files:** Create the `server.js`, `playwright_scraper.js`, and `client.html` files in the project directory. 6. **Run the Server:** Run `node server.js` in the project directory. 7. **Open `client.html`:** Open the `client.html` file in your web browser. 8. **Click "Start Scraping":** Click the "Start Scraping" button to start the scraping process. 9. **See the URLs:** The URLs will be displayed in the list as they are discovered. 10. **Click "Stop Scraping":** Click the "Stop Scraping" button to stop the scraping process. **Important Considerations and Improvements** * **Error Handling:** Implement more robust error handling in both the server and the Playwright script. Log errors to a file or a monitoring service. * **Configuration:** Use environment variables or a configuration file to store settings such as the target URL, the scraping interval, and the log file path. * **Scalability:** For large-scale scraping, consider using a message queue (e.g., RabbitMQ or Kafka) to distribute the scraping tasks across multiple workers. * **Rate Limiting:** Implement rate limiting to avoid overloading the target website. Use `page.waitForTimeout()` to pause between requests. * **User Agent:** Set a realistic user agent to avoid being blocked by the target website. * **Proxies:** Use proxies to avoid being blocked by the target website. * **Data Storage:** Store the scraped data in a database or a file. * **MCP (Management Control Protocol):** If you have specific MCP requirements, you'll need to implement the appropriate protocol and data format. This might involve adding more endpoints to the server to handle MCP commands. * **Headless Mode:** Running in headless mode is generally recommended for performance. However, some websites may require a full browser to render correctly. You can set `headless: false` in the Playwright script to see the browser window. * **Resource Management:** Be mindful of resource usage, especially memory. Close the browser and pages when they are no longer needed. * **Legal and Ethical Considerations:** Always respect the target website's terms of service and robots.txt file. Avoid scraping data that you are not authorized to access. This comprehensive example provides a solid foundation for building a web scraping application with Playwright, Node.js, Express.js, and SSE. Remember to adapt the code to your specific needs and requirements. Good luck!
MCP server for LogSeq
Interacts with LogSeq via its API.
machine-maintenance-agent
Enables agents to inspect machine status and sensor anomalies, search maintenance guidance, and create maintenance tickets through controlled MCP tools.
websearch-mcp
A self-hosted MCP server that gives AI agents deep internet research capabilities — no API keys required, powered by SearxNG, Playwright, and Docker.
feedly-mcp
Enables Claude to read your Feedly subscriptions via a local stdio server, listing folders and feeds, fetching recent articles, checking unread counts, searching for new feeds, and optionally marking items as read while keeping your API token on your machine.