Airlock

A security gate for agent extensions. Airlock audits a skill, MCP server, or tool repository before it is allowed to touch your agent. Point it at an artifact and it enumerates what the artifact is, scans it for risky shapes, reads it adversarially with a model, detonates it once inside a hardened sandbox, and returns a trust scorecard with a single verdict: allow, quarantine, or block.

Python 3.10+, MIT licensed.

The problem

Agents now install extensions the way apps install packages. A skill is a folder with a SKILL.md and some scripts; an MCP server is a one-line add. Both run with access to your tools, your files, and your environment. The registries that host them are filling faster than anyone is vetting them, and roughly one in eight public skills has been found to carry a critical vulnerability.

The extension you install today can read your ~/.aws/credentials, dump your environment to a remote host, or hide a line in its SKILL.md that tells your agent to reveal its system prompt. A careful manual review is thirty to sixty minutes of an expert reading code and declared permissions. Most people skip it and install on faith.

What Airlock does

Airlock turns that manual review into a repeatable fifteen-second gate. It runs five stages, and the gate at the end is deterministic, so a model is never the last word on whether something dangerous gets in.

1. Recon enumerates the artifact's bill of materials: its files, its SKILL.md frontmatter, its declared permissions, and its executable entrypoints.
2. The static scan matches the code and the SKILL.md against a corpus of risky shapes: credential-file reads, environment exfiltration, base64 and network pairings, reverse-shell patterns, package lifecycle hooks, and prompt-injection phrasing.
3. The adversarial judge, a Gemini model, reads the artifact for intent the patterns miss: a novel exfiltration path, an injection buried in prose, a mismatch between what SKILL.md promises and what the code does.
4. The sandbox trial detonates the artifact once inside a container with no network, a read-only root, every Linux capability dropped, and hard caps on memory, processes, and time. The environment is seeded with canary secrets. If the artifact reaches for one, Airlock sees it.
5. The trust policy fuses every signal into a grade, a tier, and a verdict. A critical finding, a canary touch, or a judged exfiltration is always a block. The model can escalate a verdict; it can never relax one.

How it works

Recon runs first and hands every later stage the same bill of materials. The static scan, the judge, and the sandbox then examine the artifact independently; none of them sees another's output, so a miss in one can't propagate. The trust policy fuses the three signals into the grade, the tier, and the verdict the scorecard reports.

The static scan and the judge are deliberately decorrelated: one matches known shapes, the other reasons about intent, and either can catch what the other misses. The sandbox turns a claim into an observation. The trust policy is plain code, so the same artifact always gets the same verdict, and a wrong or absent judgment can only ever be safe. See docs/architecture.md for the data flow and docs/security-model.md for the threat model, including how Airlock defends itself against an artifact that tries to prompt-inject the auditor.

Quickstart

conda create -n airlock python=3.12 -y
conda activate airlock
pip install -e ".[sandbox,dev]"

# Audit a known-bad example. Deterministic core, no key needed.
airlock audit examples/malicious-skill --no-llm

# Audit a safe one.
airlock audit examples/benign-skill --no-llm

The malicious example comes back grade F, blocked, with the injection line and the credential read cited. The benign one comes back grade A, allowed. Exit codes gate a pipeline: 0 allow, 1 quarantine, 2 block.

Turn on the full pipeline with a key and Docker running:

cp .env.example .env        # then set GOOGLE_API_KEY
airlock audit examples/malicious-skill

The sandbox trial detonates the example and reports the canary secrets it emitted. A Google AI Studio key enables the adversarial judge.

To see the whole flow in one place, notebook/airlock_demo.ipynb audits the malicious and benign examples, runs the eval, and falls back to the deterministic core when there is no key or Docker. On Kaggle it reads GOOGLE_API_KEY from Secrets.

Airlock also runs as an MCP server, so an agent can gate an install without shelling out:

airlock-mcp        # or: python -m airlock.mcp_server

It speaks JSON-RPC over stdio and offers one tool, audit_extension(path, use_llm?, use_sandbox?), that returns the same verdict the CLI does. The tool defaults to the deterministic core, so it answers with no key and no Docker.

Demonstrated concepts

Airlock is built for the AI Agents Intensive capstone and demonstrates the course concepts with working code, not claims.

Concept	Where it lives
Multi-agent system (ADK)	src/airlock/agents/adk_pipeline.py: a SequentialAgent of scanner, judge, and gate agents
Security features	the whole product: sandboxing, deterministic guardrails, injection defense (docs/security-model.md)
Deployability	Dockerfile and src/airlock/server.py for Cloud Run; notebook/airlock_demo.ipynb runs it end to end in Kaggle
Agent skills	Airlock audits SKILL.md skills and ships as one you can install (skill/SKILL.md)
MCP server	src/airlock/mcp_server.py exposes audit_extension over MCP so an agent can gate an install inline
Evaluation	eval/run_eval.py scores detection accuracy over a labeled corpus

Evaluation

python eval/run_eval.py          # deterministic core, reproducible offline
python eval/run_eval.py --full   # with the judge and the sandbox

The script runs the gate over eval/labels.json, a hand-built corpus of 22 fixtures: 10 malicious extensions covering distinct attack shapes (credential-file reads, environment exfiltration, a reverse shell, a curl | bash installer, an npm postinstall hook, SKILL.md prompt injection, an obfuscated loader, and an MCP server that scrapes the environment) and 12 benign utilities, several of which look suspicious but are not, like a base64 tool and a password-strength checker.

On the deterministic core the gate catches all 10 malicious fixtures and clears all 12 benign ones: recall 100%, precision 100%. Every malicious fixture is inert, targets a non-routable .invalid host, and carries a header marking it a test fixture. tests/test_corpus.py asserts each fixture still reaches its labeled verdict, so a rule change that breaks the number fails the build. Add a folder under eval/fixtures/, label it in eval/labels.json, and rerun.

Roadmap

• Live MCP-server auditing by connecting to a target server and trialing its declared tools, not just reading its manifest.
• An egress detector in the sandbox: a fake metadata endpoint and a DNS sink, so a silent exfiltrator that swallows its errors is still caught.
• The runtime side: watch an installed extension's behavior over time, not just at install.
• More languages in the scanner and the sandbox beyond Python and shell.

License

MIT. See LICENSE.