rock-mcp
Documentation crawler MCP server that crawls and indexes documentation sites so that any MCP-compatible AI can search, read, and expand on the content.
README
rock-mcp
Documentation crawler MCP server. Share a link → site gets crawled and indexed → any MCP-compatible AI can search, read, and expand on it.
No API keys. No cloud. Runs locally, stores everything in SQLite.
How it works
- Share a docs URL (
add_docs_url) - rock-mcp crawls and indexes it in the background
- Ask your AI about it — it searches, reads pages, and expands topics on demand
Tools
| Tool | Description |
|---|---|
search_web |
Search the web via DuckDuckGo — find URLs when you don't have one yet |
add_docs_url |
Crawl and index a URL (async, call immediately then search) |
search_docs |
Full-text search across all indexed docs |
get_page |
Get a specific page, chunked. Jump to a section by name. |
expand_topic |
Find all pages covering a topic |
list_sources |
List indexed sources with crawl status and staleness |
recrawl_source |
Re-crawl a source to pick up updates |
get_crawl_status |
Check crawl progress for a URL |
delete_source |
Remove a source and all its pages |
Installation
Prerequisites
- Node.js 18+
- Git
git clone https://github.com/sethwebster/rock-mcp
cd rock-mcp
npm install && npm run build
Note the full path to dist/server.js — you'll need it below.
Claude Code (CLI)
claude mcp add --scope user rock-mcp node /path/to/rock-mcp/dist/server.js
Restart Claude Code after running this. Verify with:
claude mcp list
Claude Desktop
Add to ~/Library/Application Support/Claude/claude_desktop_config.json (macOS) or %APPDATA%\Claude\claude_desktop_config.json (Windows):
{
"mcpServers": {
"rock-mcp": {
"command": "node",
"args": ["/path/to/rock-mcp/dist/server.js"]
}
}
}
Restart Claude Desktop.
Cursor
Add to .cursor/mcp.json in your project root, or ~/.cursor/mcp.json globally:
{
"mcpServers": {
"rock-mcp": {
"command": "node",
"args": ["/path/to/rock-mcp/dist/server.js"]
}
}
}
Restart Cursor.
Windsurf
Add to ~/.codeium/windsurf/mcp_config.json:
{
"mcpServers": {
"rock-mcp": {
"command": "node",
"args": ["/path/to/rock-mcp/dist/server.js"]
}
}
}
Restart Windsurf.
Codex (OpenAI CLI)
Add to ~/.codex/config.yaml (or your project's codex.yaml):
mcp_servers:
- name: rock-mcp
command: node
args:
- /path/to/rock-mcp/dist/server.js
Cline (VS Code extension)
Open VS Code settings, search for Cline MCP, and add:
{
"rock-mcp": {
"command": "node",
"args": ["/path/to/rock-mcp/dist/server.js"]
}
}
Or edit settings.json directly under the cline.mcpServers key.
Continue.dev
Add to ~/.continue/config.json under mcpServers:
{
"mcpServers": [
{
"name": "rock-mcp",
"command": "node",
"args": ["/path/to/rock-mcp/dist/server.js"]
}
]
}
Antigravity
In the agent panel, click ... → Manage MCP Servers → View raw config to open mcp_config.json. Add your server:
{
"mcpServers": {
"rock-mcp": {
"command": "node",
"args": ["/path/to/rock-mcp/dist/server.js"]
}
}
}
Save the file and restart Antigravity (or reload the window).
Firebase Studio (Project IDX)
Add to .idx/mcp.json in your workspace root (create it if it doesn't exist):
{
"mcpServers": {
"rock-mcp": {
"command": "node",
"args": ["/path/to/rock-mcp/dist/server.js"]
}
}
}
For Gemini CLI within Firebase Studio, add to .gemini/settings.json instead.
Zed
Add to ~/.config/zed/settings.json under context_servers:
{
"context_servers": {
"rock-mcp": {
"command": {
"path": "node",
"args": ["/path/to/rock-mcp/dist/server.js"]
}
}
}
}
Any MCP-compatible client
rock-mcp is a standard MCP stdio server. The binary is node /path/to/rock-mcp/dist/server.js. If your client supports MCP, point it there.
Storage
All data lives in ~/.rock-mcp/docs.db (SQLite, WAL mode). Delete it to reset everything.
Crawl defaults
- Depth: 2 hops from the root URL
- Max pages: 50 per source
- Concurrency: 5 pages at a time, 200ms between batches
- Only indexes
text/html— skips PDFs, plain text, JSON, etc. - Redirect deduplication: follows redirects and stores under the canonical URL
License
MIT
Recommended Servers
playwright-mcp
A Model Context Protocol server that enables LLMs to interact with web pages through structured accessibility snapshots without requiring vision models or screenshots.
Magic Component Platform (MCP)
An AI-powered tool that generates modern UI components from natural language descriptions, integrating with popular IDEs to streamline UI development workflow.
Audiense Insights MCP Server
Enables interaction with Audiense Insights accounts via the Model Context Protocol, facilitating the extraction and analysis of marketing insights and audience data including demographics, behavior, and influencer engagement.
VeyraX MCP
Single MCP tool to connect all your favorite tools: Gmail, Calendar and 40 more.
graphlit-mcp-server
The Model Context Protocol (MCP) Server enables integration between MCP clients and the Graphlit service. Ingest anything from Slack to Gmail to podcast feeds, in addition to web crawling, into a Graphlit project - and then retrieve relevant contents from the MCP client.
Kagi MCP Server
An MCP server that integrates Kagi search capabilities with Claude AI, enabling Claude to perform real-time web searches when answering questions that require up-to-date information.
E2B
Using MCP to run code via e2b.
Neon Database
MCP server for interacting with Neon Management API and databases
Exa Search
A Model Context Protocol (MCP) server lets AI assistants like Claude use the Exa AI Search API for web searches. This setup allows AI models to get real-time web information in a safe and controlled way.
Qdrant Server
This repository is an example of how to create a MCP server for Qdrant, a vector search engine.