Website Intelligence MCP Server
Website Intelligence MCP Server exposes 10 tools for web scraping and analysis, enabling AI agents to convert URLs to Markdown, extract metadata, detect technologies, find contacts, and perform SEO checks. It wraps a REST API and supports optional API keys.
README
Website Intelligence MCP Server
A Model Context Protocol (MCP) server that exposes 10 website intelligence tools for AI agents. Each tool wraps a REST API endpoint hosted at http://64.227.2.61/intel/.
Tools
| Tool | Description | Parameter |
|---|---|---|
url_to_markdown |
Convert any website to clean Markdown | url |
metadata_extractor |
Extract page metadata (title, description, OG tags, etc.) | url |
technology_detector |
Detect CMS, frameworks, analytics, CDN, and more | url |
contact_finder |
Find emails, phone numbers, social media links | url |
seo_analyzer |
SEO analysis — title tags, meta descriptions, headings, images | url |
robots_txt_parser |
Fetch and parse robots.txt rules |
url |
sitemap_discoverer |
Discover XML/RSS sitemaps | url |
ssl_checker |
Check SSL certificate validity, expiry, and issuer | domain |
dns_lookup |
DNS records lookup (A, AAAA, CNAME, MX, NS, TXT, etc.) | domain |
website_summary |
AI-powered summary of page content | url |
Every tool accepts an optional api_key parameter for authenticated access.
Installation
pip install web-intel-mcp-server
Usage
With an MCP client (Claude Desktop, etc.)
Add to your MCP client config:
{
"mcpServers": {
"web-intel": {
"command": "web-intel-mcp-server",
"type": "stdio"
}
}
}
Standalone
# Run with default stdio transport
web-intel-mcp-server
# Or directly with Python
python -m mcp_server
Testing with the MCP Inspector
npx @modelcontextprotocol/inspector web-intel-mcp-server
API Access & Payments
The underlying API supports Solana USDC payments for higher-rate access. If you need increased limits or a dedicated API key:
- Contact the operator with your Solana wallet address
- Send USDC on Solana to the provided wallet
- Receive your
X-API-Keyfor authenticated requests
Without an API key, endpoints may have rate limits or require one depending on server configuration.
Development
# Clone and install in editable mode
git clone https://github.com/samayhuf-star/web-intel-mcp-server.git
cd web-intel-mcp-server
pip install -e .
License
MIT
Recommended Servers
playwright-mcp
A Model Context Protocol server that enables LLMs to interact with web pages through structured accessibility snapshots without requiring vision models or screenshots.
Magic Component Platform (MCP)
An AI-powered tool that generates modern UI components from natural language descriptions, integrating with popular IDEs to streamline UI development workflow.
Audiense Insights MCP Server
Enables interaction with Audiense Insights accounts via the Model Context Protocol, facilitating the extraction and analysis of marketing insights and audience data including demographics, behavior, and influencer engagement.
VeyraX MCP
Single MCP tool to connect all your favorite tools: Gmail, Calendar and 40 more.
graphlit-mcp-server
The Model Context Protocol (MCP) Server enables integration between MCP clients and the Graphlit service. Ingest anything from Slack to Gmail to podcast feeds, in addition to web crawling, into a Graphlit project - and then retrieve relevant contents from the MCP client.
Kagi MCP Server
An MCP server that integrates Kagi search capabilities with Claude AI, enabling Claude to perform real-time web searches when answering questions that require up-to-date information.
E2B
Using MCP to run code via e2b.
Neon Database
MCP server for interacting with Neon Management API and databases
Exa Search
A Model Context Protocol (MCP) server lets AI assistants like Claude use the Exa AI Search API for web searches. This setup allows AI models to get real-time web information in a safe and controlled way.
Qdrant Server
This repository is an example of how to create a MCP server for Qdrant, a vector search engine.