agent-telemetry-mcp

An MCP server that reports what your AI agents cost: tokens, latency, dollars per request. Each caller sees only their own rows, because the customer id comes from their OAuth token rather than from a tool argument.

There is also an ADK agent that asks it questions in English.

MCP Python SDK 2.0 (protocol revision 2026-07-28), ADK 2.6, Auth0, BigQuery, Cloud Run.

Try it without installing anything

It is deployed. An unauthenticated call tells you where to authenticate:

$ curl -si https://agent-telemetry-x62fjiecda-ew.a.run.app/mcp \
    -H 'Content-Type: application/json' \
    -H 'Accept: application/json, text/event-stream' \
    -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}'

HTTP/2 401
www-authenticate: Bearer error="invalid_token", error_description="Authentication required",
  resource_metadata="https://agent-telemetry-x62fjiecda-ew.a.run.app/.well-known/oauth-protected-resource/mcp"

That last URL is the RFC 9728 document, and it is public — it names the authorization server and the scopes this resource accepts.

What it looks like

Two logins against the same deployment. demo@acme.com sees about $0.29; demo@globex.com sees about $0.49 — a bigger bill from a third as many calls, because that tenant is seeded pro-heavy. The totals creep up as you use it, since CostPlugin records each run. Then the model is taken out of the loop and the tool is called directly with a tenant named in the arguments:

{'days': 7}                       -> tenant=globex  cost=$0.4916
{'days': 7, 'tenant': 'acme'}     -> tenant=globex  cost=$0.4916
{'days': 7, 'tenant_id': 'acme'}  -> tenant=globex  cost=$0.4916

Both accounts are on a throwaway Auth0 dev tenant, and both can write: demo@acme.com / AcmeDemo-20b9c40a64-Xq7, demo@globex.com / GlobexDemo-6d9b1808cf-Kt3.

Why a server and not just ADK's BigQueryToolset

BigQueryToolset accepts an end user's token:

credentials = Credentials(token=user_token)
BigQueryToolset(credentials_config=BigQueryCredentialsConfig(credentials=credentials))

That is fine when you trust the agent. It is not enough when one agent serves many customers, because nothing checks the token — the agent uses whatever it was handed.

So the server does the checking. It validates issuer, audience and scopes, reads the customer id from the validated token, and binds that id as a query parameter. ADK's own docs say the same about their nearest equivalent:

job_labels: Note: These labels are for usage discovery and tracking purposes only and should not be used for security-sensitive decisions.

How it fits together

telemetry_analyst              picks which sub-agent answers
├── explorer_agent    ──→  BigQueryToolset   public datasets, agent's identity
└── telemetry_agent   ──→  this MCP server   your own data, your identity

CostPlugin  ──→  this MCP server  ──→  BigQuery table

CostPlugin runs on the ADK Runner, so it sees every model call from every agent in the tree. After each call it reads the token counts, prices them, and sends a row through the same authenticated connection the reads use. That is where the data in the table comes from — the system measures itself.

explorer_agent is four lines of BigQueryToolset. Read-only mode and the byte ceiling are already BigQueryToolConfig fields, so there was nothing to write.

The one rule

No tool takes a tenant argument. Not a validated one — there is no such parameter:

usage_summary          days
cost_by_model          days
slowest_invocations    days, limit
record_invocation      agent, model, prompt_tokens, output_tokens, ...

record_invocation needs telemetry:write, the reads need telemetry:read, and a token that lacks a scope does not see the tool it would have unlocked. Asking anyway returns 403 with an RFC 6750 scope hint, which the client uses to step up in one round trip. tests/test_tenancy.py fails the build if a tenant argument ever appears.

Run it

uv venv && uv pip install -e ".[dev,agent]"
cp .env.example .env          # AUTH0_DOMAIN, MCP_CANONICAL_URI, GOOGLE_CLOUD_PROJECT
                              # plus GOOGLE_API_KEY for the model

uvicorn agent_telemetry.server:create_app --factory --reload
python -m agent.main

deploy/deploy.sh provisions and deploys; the Auth0 steps that have no API are in deploy/README.md. deploy/seed.py fills a demo tenant, priced from agent/pricing.py so the demo cannot drift from the code it is demonstrating.

Tests

pytest        # 43 tests, no credentials needed

file	what it covers
test_tenancy.py	two tokens get two different slices; no tenant claim is refused
test_scope_gate.py	per-tool scopes, through the real ASGI stack
test_agent_auth.py	the client half: loopback callback, cached tokens, discovery
test_cost_plugin.py	pricing, and not double-counting streamed responses
test_wiring.py	the plugin and the server still agree on field names

Things that were not obvious

The tests all passed while the deployed server could not answer a single authenticated request. An in-memory Client(server) hands the server an object graph — no HTTP, no Host header, no ASGI receive channel. Four separate faults lived in that gap. The one I would not have guessed: streamable_http_app() defaults to a localhost-only Host allowlist, so every Cloud Run request came back 421, and only after authentication had succeeded.

scopes_supported and required_scopes are different things. The SDK builds the RFC 9728 document from required_scopes, which is the floor a token must already clear, not what a client may ask for. So the document advertised telemetry:read alone, and a client that follows the spec's scope-selection order was told telemetry:write did not exist.

The client discovers metadata on the 401 path only. A process that starts with a cached token never sees a 401, so a later 403 step-up re-authorized with no metadata in hand and fell back to {resource_server}/authorize — a URL this server does not serve. The browser opened on a 404 and the flow waited forever.

ADK's McpToolset does not work with MCP SDK 2.x. It imports mcp.shared.session, which 2.0 removed, and the package swallows the ImportError and logs it at DEBUG. What you see is cannot import name 'McpToolset'. That is why agent/toolset.py exists.

Not included

Approval prompts, the Tasks extension, MCP Apps, A2A, ADK evals. All doable, none of them make the access control any better.