Hydra ACI

<div align="center">

<img width="100%" alt="HYDRA ACI banner" src="https://capsule-render.vercel.app/api?type=waving&color=0:0B0E13,45:BE3D52,75:6366F1,100:5B9EC9&height=210&section=header&text=HYDRA%20ACI&fontSize=60&fontColor=E7E2D5&animation=fadeIn&fontAlignY=36&desc=The%20compiler%20layer%20between%20AI%20and%20the%20live%20web&descSize=19&descAlignY=60" />

<img src="assets/hydra-ascii.svg" alt="HYDRA ACI" width="560">

<img alt="Any URL in. Typed tools out." src="https://readme-typing-svg.demolab.com?font=Fira+Code&weight=600&size=22&duration=2400&pause=900&color=BE3D52&center=true&vCenter=true&width=760&lines=Any+URL+in.+Typed+tools+out.;The+other+95%25+of+the+web%2C+compiled.;mount(any_url)+%E2%86%92+call_tool(%22search%22);while+(web.stays_dark())+%7B+hydra.compile()%3B+%7D" />

The next step in the agentic web — compile any website into typed, callable tools for any agent. No MCP server on the site's side. No config on yours.

⚡ Quickstart · 🧬 How it works · 🔌 Harnesses · 🥊 Compare · 📊 Honest numbers · 🌟 Credits

<br>

</div>

---

🐙 The Mount Test

Every design decision in this repo is judged against one scenario:

You point Hydra at a URL it has never seen. Seconds later, your agent is calling typed tools against that site — search, filter, submit, whatever it actually exposes. No server was built. No config was written. No human sat in a browser.

you ──▶ hydra mount https://any-site.com
        │
        ├─ discover   H1 scout: OpenAPI · GraphQL · llms.txt · MCP apps · __NEXT_DATA__ · live XHR
        ├─ compile    canonical Web Capability IR — typed tools, ranked structured-first
        └─ serve      ▶ MCP · SDK · OpenAI · Anthropic · Gemini · LangChain
                        call_tool("search", {"q": "widgets"})   # as if the site shipped MCP all along

That is the whole product. Everything below is how it survives contact with the real web.

---

🌑 The 95% problem

Only about 5% of the web is usable by an AI agent today: the sliver that ships an OpenAPI spec, a public GraphQL endpoint, an llms.txt, a plugin manifest, or a hand-built MCP server. Everything else — the other ~95% — is dark to agents unless a human writes a scraper or sits in a browser clicking.

The industry's answer so far is "rebuild the web for agents, one endpoint at a time." That scales to the handful of destinations with the budget and incentive to opt in. It will never reach the internal tools, the legacy apps, the long tail of SaaS, the sites that already work fine for humans.

Hydra takes the other path. It reads what a site already exposes — discovered APIs, embedded app data, accessibility trees, live network traffic — and compiles that into strongly-typed, LLM-callable tools at runtime. The web does not get rebuilt. Agents learn to read it.

🔁 Hydra is the inverse of MCP. MCP asks every website to build an endpoint for agents. Hydra synthesizes the endpoint from whatever the site already has, and serves it to any harness as if a clean MCP server had existed all along.

from hydra.sdk import HydraSDK

sdk = HydraSDK()
session = await sdk.mount("https://any-site.com")          # discover + compile
print(session["capabilities"])                              # typed tools, ready to call
await sdk.call_tool(session["session_id"], "search", {"q": "widgets"})

That same mounted site works identically from Claude Code, Codex, Cursor, Gemini CLI, OpenClaw, Antigravity, LangChain, or a raw OpenAI/Anthropic tool loop. One compile, every harness — because they all consume the same capability catalog.

---

<img src="assets/hydra-hero.svg?v=8" alt="HYDRA ACI: eight tentacle arms radiating like a terminal spinner" width="1000">

🧪 Why this is a different category

Every existing approach to "agents on the web" is a local maximum. Hydra is aiming for the global one.

Tool class	What it does	Where it stops
🤖 browser-use, Playwright agents	an LLM clicks around a live browser	slow, brittle across redesigns, nothing reusable for the next agent
📄 Firecrawl, Crawl4AI, Scrapling	extract content to markdown or JSON	read-only — you get text, you cannot place the order
🔧 MCP servers	clean typed tools, where they exist	a rounding error of the web, each one hand-built per harness
📜 llms.txt, AI-plugin manifests	voluntary agent-native endpoints	a rounding error of a rounding error

The right primitive is not a better browser agent or a better scraper. It is a web capability compiler: discover the highest-signal machine surface a site already has, drop to a governed browser probe only when nothing structured exists, normalize everything into one canonical intermediate representation, expose that IR to every harness through native adapters, and remember what worked so the second visit is instant.

<p align="center"> <img src="assets/hydra-system-map.svg?v=2" alt="Hydra compiles websites and MCP apps into live tool catalogs" width="960"> </p>

---

🧬 How it works

Every session runs through a deterministic finite-state kernel, structured-first by construction. The browser is a probe, never the default driver.

%%{init: {'theme':'base','themeVariables':{'primaryColor':'#BE3D52','primaryTextColor':'#ffffff','primaryBorderColor':'#E7E2D5','lineColor':'#5B9EC9','secondaryColor':'#1e293b','tertiaryColor':'#0B0E13'}}}%%
flowchart LR
    URL([any URL]) --> K{{Orchestration Kernel<br/>INIT · DISCOVER · INGEST · EXECUTE}}
    K --> H1[H1 Scout<br/>APIs · GraphQL · llms.txt<br/>MCP · embedded data]
    K --> H8[H8 Memory<br/>prior mounts · auth shapes<br/>verified workflows]
    H1 --> P{structured<br/>surface<br/>found?}
    P -- yes --> IR
    P -- no --> PROBE[Governed browser probe<br/>H2 a11y tree · H3 XHR sniff]
    PROBE --> IR
    IR[[Canonical Web Capability IR<br/>id · schema · source · verifier · redaction]]
    IR --> AD[Native adapters]
    AD --> A1[Claude Code / Codex]
    AD --> A2[OpenAI / Anthropic / Gemini]
    AD --> A3[LangChain / MCP / ...]

Everything a site exposes becomes one stable record in the canonical Web Capability IR: an id, an input schema, a source type, an auth requirement, a side-effect class, a verifier summary, a fallback chain, and a redaction policy. A route planner always prefers the highest-signal surface (mcp > openapi > graphql > llms.txt > embedded data > observed XHR > forms > browser action), so an agent calls a real API before it ever considers clicking a button.

The eight Heads

The name is literal. Eight specialized Heads run inside an isolated browser enclave, each best-in-class at one job, composed by the kernel.

Head		Name	Role
H1	🔭	Protocol Scout	API-first discovery: OpenAPI/Swagger, AI-plugin manifests, llms.txt, GraphQL introspection, MCP configs, embedded app data (__NEXT_DATA__, Nuxt, Remix, Apollo/Relay, JSON-LD), sitemaps — before touching the DOM
H2	🌲	AXTree Compiler	Live DOM into an LLM-optimized accessibility tree, readable prose markdown, and stable fingerprints
H3	🕸️	Network Sniffer	Intercepts XHR/fetch, infers typed API schemas, merges samples across observations, redacts secrets
H4	🧬	Schema Engine	Generates typed JIT tools with token-aware budgeting and goal-relevance ranking, and emits the canonical IR
H5	🥷	Camouflage	Human-paced input (Bezier paths, WPM typing), per-session fingerprint controls, pluggable stealth backends
H6	🔐	Security Vault	AES-256-GCM, scrypt KDF, scoped secrets typed straight into the page — values never enter LLM context, traces, or archives
H7	🙋	HITL Router	Detects Turnstile, reCAPTCHA, hCaptcha, and MFA, and routes to human-in-the-loop or an opt-in solver
H8	🧠	Knowledge Archive	SQLite memory: site profiles, drift fingerprints, auth-flow shapes, verified workflows, stable-id remapping, deterministic trace replay

Full architecture: docs/architecture.md · Capability strategy: docs/capability-compiler.md

---

⚡ Quickstart

Fast local install

git clone https://github.com/Suraj1235/Hydra-aci.git
cd Hydra-aci
pip install -e .
playwright install chromium

Optional vector search for the Knowledge Archive:

pip install -e .[vector]

Python SDK

import asyncio
from hydra.sdk import HydraSDK

async def main():
    sdk = HydraSDK()

    # Mount a site, get a live typed tool catalog
    session = await sdk.mount("https://example.com", goal="find product prices")
    for cap in session["capabilities"]:
        print(cap["name"], "<-", cap["source_type"])

    # Or run a one-shot structured extraction
    data = await sdk.extract(
        "https://example.com/products",
        schema={"type": "array"},
        prompt="every product name and price",
    )
    print(data)

asyncio.run(main())

The SDK surface: mount, compile, probe, call_tool, refresh, run, scrape, crawl, extract, map, diff, replay, and a memory facade.

Drop Hydra into your coding agent with no manual config editing:

hydra-connect doctor                    # or: python -m hydra.connect doctor
hydra-setup --all --no-verify           # writes Codex, Claude Code, Cursor, Gemini CLI, OpenCode, etc.
hydra-connect verify codex --session-smoke

Harness path	Command
🩺 Public readiness check	hydra-connect doctor
🚀 Install everywhere Hydra can detect	hydra-setup --all --no-verify
🟣 Claude Code only	hydra-connect install claude-code && hydra-connect verify claude-code
🟢 Codex only	hydra-connect install codex && hydra-connect verify codex --session-smoke
🔌 Existing MCP apps	hydra-connect mcp-apps
🧹 Remove from a harness	hydra-connect uninstall claude-code

For a single harness, run hydra-connect setup claude-code or hydra-connect install claude-code && hydra-connect verify claude-code. hydra-connect doctor --json reports package health, MCP stdio readiness, configured harnesses, authorization posture, remediation, and next steps. Add --authorize when you want the installed MCP server to start with advanced browser/credential capabilities enabled. During public testing, hydra-connect uninstall claude-code removes Hydra from that harness config; pass --config when you want to target an explicit file.

🔌 Hydra also imports MCP apps you already have. On mount, Hydra scans common local MCP config files and repo manifests at .hydra/mcp-apps.json or mcp-apps.json, skips its own hydra entry, verifies upstream stdio MCP servers with tools/list, converts their tools into the same canonical IR, and exposes them beside website-compiled tools. Use ${ENV_VAR} placeholders in repo manifests so each tester supplies secrets through their shell, not through committed config. hydra-connect mcp-apps shows which apps are available and which env vars are missing. If a task is better served by an existing GitHub, database, filesystem, or SaaS MCP app, Hydra can call that app; if no MCP app exists, Hydra synthesizes the web capability itself. Secret-like environment values stay local and are redacted from capability metadata.

Or run the MCP server directly. Hydra is the MCP server. The destination site never needs one.

python -m hydra.mcp_server              # stdio
python -m hydra.mcp_http_server         # local HTTP + SSE at /mcp

{ "mcpServers": { "hydra": { "command": "python", "args": ["-m", "hydra.mcp_server"] } } }

---

🔌 Works with every harness

Because every adapter consumes the same canonical capability IR, one mounted site behaves identically everywhere it is used.

Coding agents (via hydra-connect or MCP): Claude Code, OpenAI Codex CLI, Gemini CLI, Google Antigravity, OpenCode, OpenClaw, Cursor, Cline, Windsurf. Setup recipes in docs/integrations/.

Native SDK adapters for direct model-provider users:

Provider	Import	Status
OpenAI	hydra.adapters.openai	✅ first-class
Anthropic	hydra.adapters.anthropic	✅ first-class
Google Gemini	hydra.adapters.gemini	✅ first-class
AWS Bedrock	hydra.adapters.bedrock	✅ first-class

Plus MCP, LangChain, CrewAI, AutoGen, and OpenClaw: 12 adapters, all consuming the same IR through a shared from_capabilities() contract, never re-hand-wrapped per harness. Three further adapters (Hermes, KimiClaw, AntiGravity) exist but are experimental and unverified — do not rely on them in production.

---

🥊 How Hydra compares

Hydra occupies a distinct position. browser-use is an LLM-driven browser agent. Firecrawl is a managed scrape/crawl/extract API. Lightpanda is a lightweight headless-browser runtime. Hydra's goal is orthogonal to all three: compile any site into typed callable tools so any framework can use it without a per-site integration.

Capability	Hydra	browser-use	Lightpanda	Firecrawl
Compile a site into typed callable tools	✅	❌	❌	❌
API-first, skip the browser when possible	✅	❌	❌	❌
One canonical capability IR across harnesses	✅	❌	❌	❌
Single-page markdown	🟡 beta	🟠 partial	✅	✅
Multi-page BFS crawl	🟡 beta	❌	❌	✅
Structured extraction	🟡 beta	🟠 partial	🟠 partial	✅
Multi-step agentic loop	🟡 beta	✅	❌	🟠 partial
Execution-trace replay (open source)	✅	❌	❌	❌
Native MCP server, no cloud	✅	❌	✅	✅
Encrypted credential vault	✅	🟠 partial	❌	🟠 partial
Raw browser runtime speed	API-skip avoids it	fast	fastest	cloud

The honest, cell-by-cell breakdown — including where Hydra is deliberately a category mismatch (raw runtime speed versus Lightpanda, agentic depth versus browser-use) — is in docs/comparison.md. Competitive performance numbers come only from the reproducible harness in docs/benchmarks/methodology.md, and those tables stay empty until real head-to-head runs are published.

---

📊 Status & honest numbers

Hydra is developer alpha (0.9.x), hardening toward 1.0. The ambition is large and stated plainly above. The status below is stated just as plainly, because a claim without evidence is worth nothing.

• ✅ 1,614 tests passing (5 skipped, 0 failing) across the suite.
• ✅ Real and exercised today: the canonical IR and all four compiler heads emitting it directly; the browser-as-probe orchestrator; 12 harness adapters on a shared IR contract; H8 web memory (auth-flow replay, verified-workflow replay, drift-driven stable-id rotation); the real-time MCP engine over stdio and HTTP/SSE; robots and rate-limit compliance; the encrypted vault.
• 🧪 First live benchmark data point: on a hermetic local reference app, Hydra passes 12 of 13 capability tasks (1 skipped for want of an LLM key), 100% AI-usable coverage on eligible tasks. This is Hydra-only. No competitor baseline has run yet, so it is not a competitive claim. Public summary pointers live under benchmarks/results/; per-run evidence packages are generated locally and ignored unless deliberately promoted for a release or benchmark report.
• 🎯 The number we intend to earn: move the agent-callable web from ~5% toward 60%+ of a curated sample. We do not print that number as fact until the benchmark is real on a 50-site sample with every competitor run. It is the target on the wall, not a result.

Roadmap and work-in-progress are tracked in docs/plans/.

---

🚦 Responsible use

Hydra is infrastructure for authorized agentic automation: your own apps, internal tooling, consented user automation, QA and accessibility testing, and research on systems you may access.

Defaults are conservative: robots.txt is respected and requests are rate-limited. Advanced capabilities (automated challenge solving, anti-bot fingerprint evasion, raw evaluate_js, credential injection) require a single, one-time, informed authorization granted at install or config time (hydra-authorize, --authorize on the MCP server, or HydraSDK.authorize()), persisted locally and audit-logged.

Do not use Hydra to access systems you lack permission for, to circumvent access controls, or in violation of applicable law. See AUP.md and SECURITY.md.

---

🤝 Contributing

Contributions are welcome. Read CONTRIBUTING.md first. The invariants that keep Hydra coherent (head boundaries, structured-first routing, secrets never leaving H6, determinism and replay) are documented and enforced by tests. Please keep them green.

🌟 Credits

Hydra is built in the open, and three open-source projects shaped its thinking more than any others:

• browser-use — showed how far an LLM-driven browser agent can go, and where the ceiling is. Hydra's "browser as probe, never the default driver" stance is a direct answer to lessons browser-use taught the whole field.
• Firecrawl — set the bar for developer-friendly scrape / crawl / extract APIs. Hydra's scrape, crawl, extract, and map verbs deliberately speak the dialect Firecrawl made familiar.
• Model Context Protocol — the open standard that made "typed tools for agents" a shared language. Hydra serves MCP natively, and exists to extend that promise to the sites that never built a server.

📜 License

MIT. See LICENSE.

👤 Author

Suraj Kuncham — @Suraj1235.

<div align="center">

⭐ Star the repo if you believe agents should read the web — not wait for it to be rebuilt.

The web does not need to be rebuilt for agents. Agents need to read the web.

<br>

<img src="assets/hydra-mark.svg?v=6" alt="Hydra emblem" width="88">

</div>

<img width="100%" alt="" src="https://capsule-render.vercel.app/api?type=waving&color=0:0B0E13,45:BE3D52,75:6366F1,100:5B9EC9&height=110&section=footer" />