MCP API Testing Agent

An AI-powered API testing agent that uses the Model Context Protocol (MCP) to automate API test-case generation, execution, and failure analysis.

What it does

1. Discovers endpoints from an OpenAPI/Swagger spec (via an MCP tool).
2. Generates positive and negative test scenarios for each endpoint using an LLM (LangChain + OpenAI) — valid inputs, missing required fields, wrong types, boundary values, auth failures, etc.
3. Executes each test case against the live API through MCP tools that send requests, validate responses, and analyze HTTP status codes.
4. Analyzes failures by diffing expected vs. actual responses and asking the LLM to explain why a test failed and how severe it is.
5. Reports results as a structured Markdown/JSON test report.

A FastAPI service wraps the whole pipeline so it can be triggered over HTTP (POST /agent/run) — e.g. from CI, a scheduler, or a UI — and the MCP server can also be run standalone and plugged into any MCP-compatible client (Claude Desktop, Claude Code, etc.).

Architecture

┌─────────────────────┐      OpenAPI spec / target base URL
│   FastAPI Service    │◄──────────────────────────────────
│   (api/main.py)      │
└──────────┬───────────┘
           │ triggers
┌──────────▼───────────┐
│   Testing Agent        │
│   (agent/*.py)         │
│                         │
│  1. TestGenerator       │──uses──► OpenAI (LangChain)
│     (positive/negative  │
│      scenarios)         │
│                         │
│  2. TestExecutor        │──calls──► MCP Client ──stdio──► MCP Server
│     (runs each case)    │                                  │
│                         │                          ┌───────┴────────┐
│  3. FailureAnalyzer     │                          │  MCP Tools:     │
│     (LLM explains diff) │                          │  - discover_    │
│                         │                          │    endpoints    │
│  4. ReportGenerator     │                          │  - send_request │
│     (md/json report)    │                          │  - validate_    │
└─────────────────────────┘                          │    response     │
                                                       │  - analyze_    │
                                                       │    status_code │
                                                       └────────┬───────┘
                                                                │ HTTP
                                                       ┌────────▼───────┐
                                                       │  Target API     │
                                                       │  (any REST API, │
                                                       │  e.g. sample_   │
                                                       │  target_api/)   │
                                                       └─────────────────┘

Project layout

mcp-api-testing-agent/
├── mcp_server/
│   ├── server.py               # MCP server (FastMCP) exposing the 4 tools
│   └── tools/
│       ├── discover.py         # discover_endpoints — parses OpenAPI spec
│       ├── request_tool.py     # send_request — issues HTTP calls
│       ├── validate.py         # validate_response — schema/status checks
│       └── status_analyzer.py  # analyze_status_code — status code semantics
├── agent/
│   ├── mcp_client.py           # stdio MCP client used by the agent
│   ├── test_generator.py       # LLM-based positive/negative test generation
│   ├── test_executor.py        # runs generated test cases via MCP tools
│   ├── failure_analyzer.py     # LLM explains expected-vs-actual mismatches
│   └── report_generator.py     # Markdown + JSON report writer
├── api/
│   └── main.py                  # FastAPI app: POST /agent/run, GET /agent/reports/{id}
├── schemas/
│   └── models.py                # Pydantic models shared across the app
├── sample_target_api/
│   └── demo_api.py              # tiny FastAPI service to test the agent against
├── scripts/
│   └── run_agent.py             # CLI entrypoint (no FastAPI needed)
├── reports/                     # generated test reports land here
├── Dockerfile
├── docker-compose.yml
├── requirements.txt
└── .env.example

Setup

python -m venv venv && source venv/bin/activate
pip install -r requirements.txt
cp .env.example .env   # add your OPENAI_API_KEY

Run the demo target API (a small sample API to test against)

uvicorn sample_target_api.demo_api:app --port 9000

This exposes a toy "Task Manager" API with /tasks CRUD endpoints and a generated OpenAPI spec at http://localhost:9000/openapi.json.

Run the agent via CLI

python scripts/run_agent.py --spec http://localhost:9000/openapi.json --base-url http://localhost:9000

This will generate test cases, execute them, analyze any failures, and write a report to reports/report_<timestamp>.md and .json.

Run the agent as an HTTP service

uvicorn api.main:app --port 8000

curl -X POST http://localhost:8000/agent/run \
  -H "Content-Type: application/json" \
  -d '{"spec_url": "http://localhost:9000/openapi.json", "base_url": "http://localhost:9000"}'

Run the MCP server standalone

To plug the tools into an MCP-compatible client (Claude Desktop, Claude Code, etc.) instead of the built-in agent:

python -m mcp_server.server

Then add it to your MCP client config, e.g. for Claude Desktop (claude_desktop_config.json):

{
  "mcpServers": {
    "api-testing-agent": {
      "command": "python",
      "args": ["-m", "mcp_server.server"],
      "cwd": "/absolute/path/to/mcp-api-testing-agent"
    }
  }
}

Run everything with Docker

docker compose up --build

This starts the demo target API, the MCP-backed testing agent FastAPI service, and mounts ./reports so generated reports are available on the host.

Sample report output

# API Test Report — 2026-02-03T10:15:00

Target: http://localhost:9000
Total: 18   Passed: 15   Failed: 3   Pass rate: 83%

## Failures

### POST /tasks — missing required field "title" (negative test)
Expected: 422 Unprocessable Entity
Actual:   500 Internal Server Error
Analysis: The endpoint does not validate the request body before hitting the
database layer, so a missing "title" causes an unhandled exception instead
of a client-error response. Severity: High — indicates missing input
validation.

Notes on adapting this to a real project

• Swap sample_target_api/ for your real service, or point --spec / spec_url at any live OpenAPI/Swagger JSON endpoint.
• test_generator.py's prompt can be extended with domain rules (e.g. required auth headers, rate limits, tenant IDs).
• For CI, run scripts/run_agent.py as a pipeline step and fail the build if report["summary"]["failed"] > 0.