screaming-frog-mcp

MCP server that lets Claude run Screaming Frog SEO Spider headless crawls, export data, and manage crawl storage — without anyone opening the GUI.

Type a URL into Claude. Screaming Frog runs in the background. You get the data back. That's it.

Forked from bzsasson/screaming-frog-mcp v0.1.0 with bug fixes. The original had issues that made it unusable in practice — pipe deadlocks that hung crawls, false GUI detection that blocked everything after the first run, a delete command that could wipe your entire crawl database. All fixed.

---

What's fixed

Bug	Fix
Pipe deadlock	stdout/stderr redirected to log files instead of PIPE. Crawls no longer hang when SF produces large output.
GUI detection	Uses psutil instead of ps aux. Works on Mac and Windows. Headless CLI processes no longer get mistaken for the GUI.
Stale crawl cleanup	SF leaves a temp crawl.seospider file inside its own app bundle when a crawl gets interrupted. Every crawl after that fails. Now auto-cleaned before each run.
Delete safety	delete_crawl(".") used to resolve to the root data directory and wipe everything. Fixed.
Export dir leak	Failed exports left temp directories on disk. Now cleaned up.
Input validation	Stricter character allowlists for CLI arguments and db_id.

---

Requirements

• Screaming Frog SEO Spider with a paid license — headless crawls require a license
• Python 3.10+
• uv (recommended) or pip

---

Installation

Mac

uvx --from git+https://github.com/marykovziridze/screaming-frog-mcp screaming-frog-mcp

Add to ~/Library/Application Support/Claude/claude_desktop_config.json:

{
  "mcpServers": {
    "screaming-frog": {
      "command": "uvx",
      "args": ["--from", "git+https://github.com/marykovziridze/screaming-frog-mcp", "screaming-frog-mcp"]
    }
  }
}

Windows

Install uv first:

powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

Add to C:\Users\[name]\AppData\Roaming\Claude\claude_desktop_config.json:

{
  "mcpServers": {
    "screaming-frog": {
      "command": "uvx",
      "args": ["--from", "git+https://github.com/marykovziridze/screaming-frog-mcp", "screaming-frog-mcp"],
      "env": {
        "SF_CLI_PATH": "C:\\Program Files (x86)\\Screaming Frog SEO Spider\\ScreamingFrogSEOSpiderCli.exe"
      }
    }
  }
}

Restart Claude Desktop after editing the config.

---

Tools

Tool	What it does
sf_check	Verify SF is installed and licensed
crawl_site	Start a headless crawl
crawl_status	Check crawl progress
list_crawls	List saved crawls in SF's database
export_crawl	Export crawl data as CSV
read_crawl_data	Read and filter exported CSV data
delete_crawl	Delete a saved crawl
storage_summary	Show disk usage of crawl storage

---

Configuration

Variable	Default	Notes
SF_CLI_PATH	Mac: auto-detected	Set manually on Windows or custom installs

---

Known limitations

• Windows stale crawl path — auto-cleanup works on Mac. On Windows, if crawls fail after an interruption, check for a crawl.seospider file in your SF install directory and delete it manually.
• No crawl progress percentage — SF's headless CLI doesn't report progress mid-crawl. You know when it starts and when it finishes.
• Large sites — tested on sites up to ~160 pages. Not stress-tested on 10k+ page sites.

---

License

MIT — see LICENSE

---

Credits

Original MCP server by Boaz Sasson.