MCP Servers

spider-cloud-mcp

MCP server for the Spider web crawling and scraping API, enabling AI agents to crawl, scrape, search, and extract web data.

README

spider-cloud-mcp

MCP server for the Spider web crawling and scraping API. Crawl, scrape, search, and extract web data for AI agents, RAG pipelines, and LLMs.

Setup

1. Get your API key

2. Configure your MCP client

Claude Desktop

Add to your claude_desktop_config.json:

{
  "mcpServers": {
    "spider": {
      "command": "npx",
      "args": ["-y", "spider-cloud-mcp"],
      "env": {
        "SPIDER_API_KEY": "your-api-key"
      }
    }
  }
}

Claude Code

claude mcp add spider -- npx -y spider-cloud-mcp

Then set your API key in the environment or .env file:

SPIDER_API_KEY=your-api-key

Cursor

Add to your Cursor MCP settings:

{
  "mcpServers": {
    "spider": {
      "command": "npx",
      "args": ["-y", "spider-cloud-mcp"],
      "env": {
        "SPIDER_API_KEY": "your-api-key"
      }
    }
  }
}

Tools

Core Tools

Tool	Description
`spider_crawl`	Crawl a website and extract content from multiple pages
`spider_scrape`	Scrape a single page (no crawling)
`spider_search`	Search the web with optional page content fetching
`spider_links`	Extract all links from a page
`spider_screenshot`	Capture page screenshots
`spider_unblocker`	Access bot-protected content with anti-bot bypass
`spider_transform`	Transform HTML to markdown/text/other formats
`spider_get_credits`	Check your credit balance

AI Tools (Subscription Required)

These tools require an active AI subscription plan.

Tool	Description
`spider_ai_crawl`	AI-guided crawling with natural language prompts
`spider_ai_scrape`	Extract structured data using plain English
`spider_ai_search`	AI-enhanced semantic web search
`spider_ai_browser`	Automate browser interactions with natural language
`spider_ai_links`	Intelligent link extraction and filtering

Examples

Crawl a website to markdown

Use spider_crawl to crawl https://example.com with limit 10 and return_format "markdown"

Search the web

Use spider_search to search for "latest AI research papers" with fetch_page_content true and num 5

AI-powered extraction

Use spider_ai_scrape on https://news.ycombinator.com with prompt "Extract all article titles, URLs, points, and comment counts as structured JSON"

Check credits

Use spider_get_credits to check my balance

API Reference

Full API documentation: spider.cloud/docs/api

License

MIT

Recommended Servers

playwright-mcp

A Model Context Protocol server that enables LLMs to interact with web pages through structured accessibility snapshots without requiring vision models or screenshots.

Official

Featured

TypeScript

Magic Component Platform (MCP)

An AI-powered tool that generates modern UI components from natural language descriptions, integrating with popular IDEs to streamline UI development workflow.

Audiense Insights MCP Server

Enables interaction with Audiense Insights accounts via the Model Context Protocol, facilitating the extraction and analysis of marketing insights and audience data including demographics, behavior, and influencer engagement.

VeyraX MCP

Single MCP tool to connect all your favorite tools: Gmail, Calendar and 40 more.

Official

Featured

Local

graphlit-mcp-server

The Model Context Protocol (MCP) Server enables integration between MCP clients and the Graphlit service. Ingest anything from Slack to Gmail to podcast feeds, in addition to web crawling, into a Graphlit project - and then retrieve relevant contents from the MCP client.

Official

Featured

TypeScript

Kagi MCP Server

An MCP server that integrates Kagi search capabilities with Claude AI, enabling Claude to perform real-time web searches when answering questions that require up-to-date information.

Official

Featured

Python

E2B

Using MCP to run code via e2b.

Official

Featured

Neon Database

MCP server for interacting with Neon Management API and databases

Official

Featured

Exa Search

A Model Context Protocol (MCP) server lets AI assistants like Claude use the Exa AI Search API for web searches. This setup allows AI models to get real-time web information in a safe and controlled way.

Official

Featured

Qdrant Server

This repository is an example of how to create a MCP server for Qdrant, a vector search engine.

Official

Featured