MCP Servers

@aegis-ai/mcp-scraper

Enables intelligent web scraping with support for static pages, JavaScript-rendered SPAs, and natural language data extraction.

README

@aegis-ai/mcp-scraper

MCP server for intelligent web scraping. Supports static pages, JavaScript-rendered SPAs (via Playwright), and natural language data extraction.

Install

# Add to Claude Desktop config (~/.claude/claude_desktop_config.json):
{
  "mcpServers": {
    "scraper": {
      "command": "bunx",
      "args": ["@aegis-ai/mcp-scraper"]
    }
  }
}

# Or run directly:
bunx @aegis-ai/mcp-scraper

Tools

scrape

Fetch a web page and return clean text content.

{
  "url": "https://news.ycombinator.com",
  "js": false,
  "selector": ".titleline",
  "headers": { "Cookie": "session=abc" }
}

extract

Scrape a page and extract structured data using natural language.

{
  "url": "https://example.com/products",
  "prompt": "Get all product names, prices, and ratings",
  "js": true
}

batch_scrape

Scrape multiple URLs with rate limiting (1s between requests).

{
  "urls": ["https://example.com/page1", "https://example.com/page2"],
  "js": false
}

Features

Static page fetching (fast, no browser needed)
JavaScript rendering via Playwright (for SPAs)
CSS selector targeting
Custom headers support
Automatic HTML stripping
Content truncation to avoid context overflow
Rate-limited batch scraping

License

MIT - AEGIS AI Cooperative

Recommended Servers

playwright-mcp

A Model Context Protocol server that enables LLMs to interact with web pages through structured accessibility snapshots without requiring vision models or screenshots.

Official

Featured

TypeScript

Magic Component Platform (MCP)

An AI-powered tool that generates modern UI components from natural language descriptions, integrating with popular IDEs to streamline UI development workflow.

Audiense Insights MCP Server

Enables interaction with Audiense Insights accounts via the Model Context Protocol, facilitating the extraction and analysis of marketing insights and audience data including demographics, behavior, and influencer engagement.

VeyraX MCP

Single MCP tool to connect all your favorite tools: Gmail, Calendar and 40 more.

Official

Featured

Local

graphlit-mcp-server

The Model Context Protocol (MCP) Server enables integration between MCP clients and the Graphlit service. Ingest anything from Slack to Gmail to podcast feeds, in addition to web crawling, into a Graphlit project - and then retrieve relevant contents from the MCP client.

Official

Featured

TypeScript

Kagi MCP Server

An MCP server that integrates Kagi search capabilities with Claude AI, enabling Claude to perform real-time web searches when answering questions that require up-to-date information.

Official

Featured

Python

E2B

Using MCP to run code via e2b.

Official

Featured

Neon Database

MCP server for interacting with Neon Management API and databases

Official

Featured

Exa Search

A Model Context Protocol (MCP) server lets AI assistants like Claude use the Exa AI Search API for web searches. This setup allows AI models to get real-time web information in a safe and controlled way.

Official

Featured

Qdrant Server

This repository is an example of how to create a MCP server for Qdrant, a vector search engine.

Official

Featured