BAIC Data Layer MCP Server

An MCP server over the BAIC Data Layer: sources, knowledge bases, knowledge clusters, the three data agents, and transformation pipelines.

API behaviour is documented in DL-API-REFERENCE.md, verified live against two environments. Read that first if anything here surprises you — several endpoints are named misleadingly.

SESSION-LOG.md records how this was built: the platform behaviours found, the bugs fixed along the way, claims that were later corrected, and the open items.

Setup

cp .env.example .env      # then fill in credentials + hosts
uv sync

.env is gitignored (as is any .env* except the example). Nothing environment-specific is hardcoded: the tenant and user ids are derived from the access token at runtime, because the platform uses different tenant ids on different endpoints and they vary per deployment.

Pointing at a different environment

Edit .env — the hosts and credentials there are all that select a deployment:

BAIC_UI_URL  BAIC_BACKEND_URL  BAIC_USER_MGMT_URL  BAIC_USER_NAME  BAIC_USER_PASSWD

Keeping spare copies (.env-udl-demo, .env-poc-farmers) and copying one over .env works fine; all .env* files are gitignored except .env.example.

.env is optional when the variables are supplied directly, which is how Docker (--env-file) and fastmcp.cloud work — the image ships no dotenv file on purpose. Missing values fail at startup naming the variable.

whoami reports the backend host, tenant and user, so you can always confirm where you are pointed before running anything.

Running

Built on FastMCP. Two transports, chosen by BAIC_TRANSPORT:

uv run baic-dl-mcp                          # stdio (default) - what Claude Desktop drives
BAIC_TRANSPORT=http uv run baic-dl-mcp      # HTTP on :8000

Claude Desktop — local

{
  "mcpServers": {
    "baic-dl": {
      "command": "uv",
      "args": ["--directory", "/absolute/path/to/BAIC-DL-MCP", "run", "baic-dl-mcp"]
    }
  }
}

Claude Desktop — Docker

docker build -t baic-dl-mcp:latest .

{
  "mcpServers": {
    "baic-dl": {
      "command": "docker",
      "args": ["run", "-i", "--rm",
               "--env-file", "/absolute/path/to/BAIC-DL-MCP/.env",
               "baic-dl-mcp:latest"]
    }
  }
}

-i is required — stdio needs stdin held open. Credentials are passed at run time via --env-file and never baked into the image.

fastmcp.cloud

fastmcp.json points at server.py:mcp and is ready to deploy. Connect the repo in fastmcp.cloud, then set these as environment variables in the cloud UI — never commit them:

BAIC_USER_NAME  BAIC_USER_PASSWD  BAIC_UI_URL  BAIC_BACKEND_URL  BAIC_USER_MGMT_URL

Root server.py is a path shim, not a second copy of the server: hosted runners load that file directly rather than importing the installed package, so the relative imports inside src/baic_dl_mcp/ need src on the path first.

HTTP, for a shared deployment

docker compose up -d          # serves http://localhost:8000/mcp

Both paths are verified: 36 tools listed and tools invoked successfully over container stdio and container HTTP.

Start with whoami to confirm which environment you're pointed at.

Tools

36 tools.

Connectivity

Tool	Purpose
whoami	Confirm connectivity and show which environment, tenant, and user are in play

Sources

Tool	Purpose
list_sources	List configured data sources (source connections), one page at a time
get_source	Full detail for one source, including its connection config (secrets masked)
list_connector_types	The connector engines this platform supports
create_snowflake_source	Create a new Snowflake source connection
create_postgres_source	Create a new Postgres source connection

Ingestion

Tool	Purpose
list_available_tables	List the tables a source exposes, before importing any of them
list_destinations	List destinations - the vector stores or warehouses that ingestion writes into
import_source_metadata	Crawl a source's metadata into a new knowledge base
import_and_wait	Import a source's metadata into a knowledge base and wait for it to finish
get_ingestion_task	Check the status of a metadata import started by import_source_metadata

Knowledge bases

Tool	Purpose
list_knowledge_bases	List knowledge bases (KBs) - the metadata imported via Data Ingestion, one page at a time
list_kb_tables	List the tables inside a knowledge base, one page at a time
get_table_details	Table descriptions and column-level detail for one or more tables
get_table_relationships	Discovered foreign-key style relationships between tables in a knowledge base

Knowledge resources

Tool	Purpose
list_golden_sql	List the Golden SQL examples attached to a knowledge base
add_golden_sql	Save a SQL query as a Golden SQL example on a knowledge base
delete_golden_sql	Delete a Golden SQL example. Requires BAIC_ALLOW_DESTRUCTIVE=true
get_kb_instructions	Read the instructions attached to a knowledge base, with all versions
update_kb_instructions	Overwrite the active instructions on a knowledge base. Replaces, does not append

Knowledge clusters

Tool	Purpose
list_knowledge_clusters	List knowledge clusters (KCs) - the groupings of knowledge bases used by the agents
get_knowledge_cluster	KC detail: member knowledge bases, their engines, and the KC instructions
get_kc_instructions	Read a knowledge cluster's instructions, with all versions
update_kc_instructions	Overwrite a knowledge cluster's active instructions. Replaces, does not append
create_knowledge_cluster	Create a knowledge cluster from knowledge bases, optionally with instructions
check_kc_federation	Check whether a set of knowledge bases could legally share one knowledge cluster
delete_knowledge_cluster	Delete a knowledge cluster. Requires BAIC_ALLOW_DESTRUCTIVE=true

Agents

Tool	Purpose
discover_tables	Ask the Data Discovery agent which tables and columns are relevant to a question
ask_data_analyst	Ask the Data Analyst agent a question about a knowledge cluster
generate_pipeline_sql	Ask the Data Engineering agent to write pipeline SQL
approve_table_selection	Confirm the table selection and generate the SQL. Step 2 of 2

Pipelines

Tool	Purpose
list_pipelines	List data pipelines, optionally only those in one knowledge cluster
create_pipeline	Create a data pipeline
get_pipeline_sql	Read a pipeline's saved SQL and visual pipeline graph. Defaults to the active version
save_pipeline_sql	Save SQL and/or the visual pipeline graph onto a pipeline version. Write-only
delete_pipeline	Delete a pipeline. Requires BAIC_ALLOW_DESTRUCTIVE=true

Two things worth knowing before you use it

Saving pipeline SQL is a write, not a review

Pass save_to_version_id to generate_pipeline_sql or approve_table_selection and the SQL is generated and saved in that one call. You get back a saved receipt plus a short sql_preview; the full SQL body and graph are deliberately not returned.

That shape is intentional. Returning 1-2 kB of SQL to the model invites it to re-read the SQL, call get_table_details to check the column names, and then write several paragraphs explaining the query and flagging joins for review. None of that is wanted — the SQL comes from the platform's own engineering agent, generated from the knowledge cluster's metadata and instructions. With nothing to summarise, there is nothing to summarise.

save_pipeline_sql behaves the same way if you call it directly: compact receipt, no SQL echo. Its description tells the model not to validate, not to reformat, and not to restate. Use get_pipeline_sql when you actually want the SQL back.

Two related things worth knowing:

• Empty columns from get_table_details is normal, not a verification failure. It means enrichment has not run on that table. The response now says so explicitly, because reading it as "I cannot verify this" was what triggered the hedging in the first place. The agents read the source schema directly and do not depend on that endpoint.
• Branch on outcome, don't assume a review. generate_pipeline_sql usually returns review_required with numbered_tables for a human to pick from, but it sometimes returns sql_generated straight away, and occasionally no_sql_returned.

ask_data_analyst retries on empty results, on purpose

Snowflake execution is intermittent: the identical request returned data in roughly 3 of 14 measured attempts. The UI behaves the same way, so it is not a client problem. Postgres is reliable — 5/5 on the same kind of query.

Worse, a failed execution is reported as IWX-AI-SUCCESS-001 with an empty result and the answer "The query returned no results", so on Snowflake an empty result is indistinguishable from a failure. (Tell-tale: a genuinely empty table returns one row, [{"count": 0}]; a failed execution returns zero rows, [].)

So the tool retries up to max_attempts (default 3) and reports per-attempt outcomes. Retrying often recovers but is not dependable. If every attempt is empty you get an execution_warning — do not relay that to a user as "there is no data". Evidence in DL-API-REFERENCE.md §5.1.

The BAIC DataLayer skill

skills/baic-datalayer/ covers the whole Data Layer, organised the way the UI is — Manage Connector, Data Ingestion, Data Visualization, and the three Data Agents — with a table mapping each UI surface to its tools, plus the flows for building a pipeline, asking a question, and onboarding a source.

It leads with three rules that override default assistant behaviour, because each one breaks a scripted demo:

1. Pass user wording through verbatim. The knowledge cluster's instructions already supply grain, dedup keys, null handling and dialect, so an "improved" prompt generates different SQL than the one you rehearsed.
2. Never read, validate or summarise generated SQL. No checking column names via get_table_details, no explaining the design, no flagging joins for review. Saves are reported in one line.
3. Chain to the next tool call instead of narrating.

reference/quirks.md carries the platform behaviours worth not re-deriving — the intermittent Snowflake execution, the review gate that does not always fire, empty columns being normal.

Adding it to Claude Desktop

The skill links to the API reference, which Desktop cannot reach because uploads have no access to repo files. Build the self-contained bundle first — it copies the referenced docs inside and rewrites the links:

uv run python scripts/package.py
# -> dist/baic-datalayer-skill.zip

Then in Claude Desktop:

1. Settings → Capabilities → Skills (on some builds: Settings → Features → Skills)
2. Upload skill and choose dist/baic-datalayer-skill.zip
3. Confirm baic-datalayer is listed and enabled
4. Start a new conversation — skills are picked up per conversation, not retroactively
5. Check it loaded by asking something that should trigger it, e.g. "list the knowledge clusters in baic"

The zip has the skill folder as its root, which is the layout Desktop expects:

baic-datalayer/
  SKILL.md
  reference/quirks.md
  reference/api-reference.md    <- copied in, links rewritten

Rebuild and re-upload whenever the skill or the tool set changes.

Claude Code does not load this skill in this repo, deliberately: it lives in skills/, not .claude/skills/, because it tells an assistant how to drive the Data Layer, which is not what you are doing when you are editing the server. To use it from Claude Code in another project, copy skills/baic-datalayer/ into that project's .claude/skills/, or unzip the bundle there.

It stays environment-neutral

The skill names no deployment: no hostnames, user emails, ids, or row counts. It tells the model to call whoami first and to discover ids at run time, because all of those change when you point .env somewhere else.

DL-API-REFERENCE.md is deliberately not bundled into the skill. It is a human record of building this server — raw endpoints, one deployment's hosts, ids and counts — and shipping it made the skill assert things that become false the moment you switch environment. The skill drives MCP tools, not HTTP, so it does not need it. That dropped the bundle from 31 KB to 5 KB.

scripts/package.py fails the build if the bundle mentions a specific host, email, or UUID, so this cannot creep back in.

Keeping it honest

uv run python scripts/check_skill.py

Fails if the skill names a tool that does not exist, and warns if a tool is never mentioned. A skill that points at a renamed tool sends the model down a dead end, so this is checked rather than trusted.

It is a nudge, not a guarantee

Skills and tool descriptions steer; they do not enforce. The enforcement is structural:

• Pass save_to_version_id so SQL is saved server-side and never returned to the model. SQL that is not in the response cannot be summarised.
• save_pipeline_sql returns a compact receipt, not the SQL body.
• get_table_details labels an empty columns list as expected, so it stops reading as a verification failure.

instruction and question are documented as verbatim in the tool descriptions and in the server-level instructions too, so the rule still holds when the skill is not loaded.

Guard rails

Two flags in .env, both default-off:

• BAIC_ALLOW_DESTRUCTIVE — gates delete_pipeline, delete_golden_sql, delete_knowledge_cluster
• BAIC_ALLOW_AUTO_APPROVE — reserved for chaining the pipeline review gate automatically

create_knowledge_cluster also refuses knowledge bases that would need federation. All structured KBs in a cluster must share one engine and one account — the platform cannot query across them, and the API will not stop you. Unstructured (document) KBs are exempt.

Packaging and release

uv run python scripts/package.py        # build skill bundle + validate all targets
uv run python scripts/package.py --check # validate only

Three deployment targets are kept working at once, and the validator fails if any breaks:

Target	Entry	Transport
Local	baic-dl-mcp console script	stdio
Docker	Dockerfile / docker-compose.yml	stdio or http
fastmcp.cloud	fastmcp.json → server.py:mcp	http

Before a demo or a release, run the sequence in the package-baic-mcp skill — it regenerates the README tool table, checks the skill for stale tool names, builds the bundle, rebuilds the image, and finishes with a live smoke test against the configured environment.

Credentials never enter an artefact: .gitignore covers .env* (except .env.example), .dockerignore keeps them out of the image, and both are asserted by the validator. Docker and fastmcp.cloud take them as run-time environment variables.

Layout

src/baic_dl_mcp/
  config.py            env loading
  client.py            auth, token refresh, envelope normalisation, paging
  domain.py            the operations, and the reasons behind each workaround
  server.py            FastMCP tool surface
server.py              path shim for hosted runners (fastmcp.cloud) - no logic
fastmcp.json           fastmcp.cloud deployment config
Dockerfile             stdio + http, credentials injected at run time
docker-compose.yml     HTTP deployment
scripts/
  gen_readme_tools.py  regenerates the Tools table from the running server
  check_skill.py       fails if the skill names a tool that does not exist
  package.py           builds the Desktop skill bundle, validates all targets
skills/
  baic-datalayer/      how to drive the Data Layer - ships to Desktop, not loaded here
.claude/skills/
  package-baic-mcp/    the release workflow, used while working in this repo

The transport layer is deliberately thin: config/client/domain know nothing about MCP, which is why swapping the server from the low-level SDK to FastMCP touched only server.py.

The Tools table above is generated from the running server, so it cannot drift out of date:

uv run python scripts/gen_readme_tools.py

It fails if a tool exists that the table does not list, or vice versa. Run it after adding a tool.

A note on tool schemas

Tools are registered with FastMCP's decorator and no hand-rolled wrapper. An earlier version wrapped each tool in *args, **kwargs and copied __annotations__ but not __signature__; MCP introspects the callable it is handed, so every tool advertised two bogus string parameters named args and kwargs, and clients sent those instead of the real arguments. If you add a wrapper here, set __signature__ — or better, don't wrap.