MCP Client Compatibility Probe

Small diagnostic MCP server for checking what MCP clients actually support.

The server is intentionally dependency-free and split into transport-neutral core logic plus a local stdio adapter. A future HTTP adapter can reuse probe-core.mjs for ChatGPT Web, OpenAI API, or remote MCP testing.

The intended workflow is AI-assisted: point the assistant/client you want to test at this repository and let it run the probe, inspect the trace, create a result file, and prepare a commit. In practice that is a single prompt.

The current informal client support overview lives in CLIENT-MATRIX.md. Detailed test design and result templates live in TESTPLAN.md.

Quick Start For Testers

Option A: One Prompt

Start the assistant or client you want to test, in a directory it is allowed to write to, and give it this:

Clone https://github.com/kascada/mcp-client-compat-probe.git, then read PROMPT.md from that clone and follow the prompt inside it. You are the client under test.

That is the whole setup. From there the assistant clones the repo, runs the smoke test, registers the probe as a local MCP server, runs the probe interactions, inspects the trace, and writes the result file. It comes back to you only for the things it genuinely cannot do itself: restarting the client so it picks up the MCP config, invoking anything the client exposes only as a user action, and approving the push or pull request.

This assumes a client that can run shell commands and read local files, such as Claude Code, Codex CLI, OpenCode or Cursor. If yours cannot, use Option B.

Option B: Step By Step

The same test, spelled out. Use this if your client cannot clone on its own, or if you want to see what Option A will do before you run it.

1. Clone this repository.

   git clone https://github.com/kascada/mcp-client-compat-probe.git
   cd mcp-client-compat-probe
   

   HTTPS is recommended for most testers because it works without a configured SSH key. If you already use GitHub over SSH, this is equivalent:

   git clone git@github.com:kascada/mcp-client-compat-probe.git
   cd mcp-client-compat-probe
   

2. Open the cloned directory in the MCP-capable assistant/client you want to test.

3. Ask the assistant to run PROMPT.md, for example: Run PROMPT.md. If the assistant cannot read local files, paste the full content of PROMPT.md instead.

4. Follow only the explicit prompts for client restart, MCP setup confirmation, and push/PR approval.

The assistant should handle the rest:

• run npm run smoke
• help configure the local stdio MCP server if needed
• run the probe interactions
• inspect the trace file
• write results/<client>-<username>-<date>.md
• stage and commit only that result file

Do not commit full trace files by default. Result files should include only small redacted excerpts.

Contributing A Result

This repository is public, which means anyone can read and clone it, but not push to it. Cloning does not create a fork and grants no write access, so contributing a result goes through a pull request from your own fork. The assistant can do this for you; the manual equivalent is:

gh repo fork --remote                                   # your own fork, no permissions needed here
git switch -c probe-result-<client>-<username>
git add results/<client>-<username>-<date>.md           # only the result file
git commit -m "Add <client> probe result <username> <date>"
git push -u origin probe-result-<client>-<username>     # pushes to your fork
gh pr create --repo kascada/mcp-client-compat-probe

Use your GitHub account name as <username>, so the result is attributable in the shared collection.

If you cannot or do not want to open a pull request, both of these are fine too:

• Open an issue and attach the result file.
• Send the result file to the repository author directly, together with the client version, the operating system, and your MCP config with secrets removed.

Files

mcp-probe/
  README.md              # quickstart and feature overview
  CLIENT-MATRIX.md       # informal client support matrix
  PROMPT.md              # assistant prompt for running and recording tests
  TESTPLAN.md            # repeatable client test plan
  probe-core.mjs          # JSON-RPC handlers and probe tools
  stdio-server.mjs        # local stdio transport
  opencode.json           # isolated OpenCode test config
  package.json            # npm scripts, no dependencies
  results/                # contributed client observations
  scripts/smoke-stdio.mjs # direct stdio smoke test

Probe Coverage

Implemented MCP methods:

• server/discover
• legacy initialize fallback response
• tools/list
• tools/call
• resources/list
• resources/read
• resources/templates/list
• prompts/list
• prompts/get
• stub subscriptions/listen

Tools:

• echo_meta: returns received arguments, _meta, client capabilities, and transport observations.
• structured_result: returns text plus structuredContent matching an outputSchema.
• create_handle: creates an explicit state handle.
• use_handle: uses a handle from create_handle.
• needs_form_input: returns resultType: "input_required" until retried with inputResponses.
• tool_error: returns a tool execution error via isError: true.
• resource_link_result: returns a resource_link content item.
• search: ChatGPT-compatible search stub.
• fetch: ChatGPT-compatible fetch stub.

Smoke Test

Run from this directory:

npm run smoke

Or without npm:

node scripts/smoke-stdio.mjs

The smoke test writes its trace to:

/tmp/mcp-probe-smoke.ndjson

Trace Log

The server never writes diagnostics to stdout, because stdout must contain only MCP JSON-RPC messages. Diagnostics go to stderr and the trace file.

Default trace path:

/tmp/mcp-probe.ndjson

OpenCode trace path from opencode.json:

/tmp/mcp-probe-opencode.ndjson

Each line is JSON with:

• ts: timestamp
• pid: server process ID
• direction: in or out
• payload: JSON-RPC payload

Test With OpenCode

This directory contains an isolated opencode.json:

{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "probe": {
      "type": "local",
      "command": ["node", "stdio-server.mjs"],
      "cwd": ".",
      "enabled": true,
      "timeout": 10000,
      "environment": {
        "MCP_PROBE_TRACE": "/tmp/mcp-probe-opencode.ndjson"
      }
    }
  }
}

Start OpenCode from this directory so it loads the local config:

opencode

Then ask:

Nutze das probe echo_meta Tool und zeige mir, welche MCP-Metadaten du gesendet hast.

Additional useful prompts:

Nutze probe structured_result mit label opencode.

Erzeuge mit probe create_handle ein Handle fuer confluence und nutze es danach mit probe use_handle fuer die Query release notes.

Teste probe needs_form_input fuer topic OpenCode Elicitation.

Nutze probe search fuer query probe und danach probe fetch fuer das erste Ergebnis.

Interpret the trace:

• server/discover present: modern MCP discovery probe is used.
• initialize present: legacy handshake path is used.
• _meta.io.modelcontextprotocol/protocolVersion present: per-request protocol version is sent.
• _meta.io.modelcontextprotocol/clientCapabilities.elicitation present: client declares elicitation support.
• resources/list or prompts/list present: client actively queries non-tool primitives.
• Retry after input_required: MRTR/Elicitation flow is handled.

OpenCode reads config at startup. Restart OpenCode after changing opencode.json or server files.

Test With Codex CLI Or ChatGPT Desktop

The same local stdio server can be used by Codex CLI, ChatGPT Desktop app, and Codex IDE extension because they support local MCP servers.

Example Codex CLI registration from this directory:

codex mcp add probe --env MCP_PROBE_TRACE=/tmp/mcp-probe-codex.ndjson -- node stdio-server.mjs

Then use /mcp in Codex to inspect active servers and ask for the same probe tools as above.

For ChatGPT Desktop app, add a new MCP server in Settings with:

• Name: probe
• Type: STDIO
• Command: node
• Args: absolute path to stdio-server.mjs
• Environment: MCP_PROBE_TRACE=/tmp/mcp-probe-chatgpt-desktop.ndjson

ChatGPT Web And OpenAI API Path

ChatGPT Web cannot directly start a local stdio server or read local Codex/OpenCode configuration. For ChatGPT Web or OpenAI API testing, add a remote HTTP adapter later.

The current design keeps that path open:

• probe-core.mjs has no stdio-specific behavior.
• stdio-server.mjs only adapts newline-delimited JSON-RPC to handleJsonRpc.
• A future http-server.mjs can call the same handleJsonRpc and pass HTTP headers in the transport object.
• The existing search and fetch tools already follow the simple ChatGPT-compatible shape with structuredContent and URL-backed results.

HTTP-specific checks to add later:

• MCP-Protocol-Version, Mcp-Method, Mcp-Name
• static/Bearer headers
• OAuth behavior
• x-mcp-header from tool parameters
• Streamable HTTP response behavior