databricks_ops_mcp

A governed MCP server exposing Databricks operations as tools — jobs orchestration, SQL execution, notebook creation, Unity Catalog governance, lineage, clusters, and DLT pipelines. Built as a reuse template: same pattern as a SharePoint/Jira MCP server, pointed at the Databricks REST API.

Tools (17)

Category	Read-only	Write-gated
Jobs	list_jobs, get_run_status, list_failed_runs	run_job, repair_run
SQL	preview_table	execute_sql (DML/DDL only; SELECT is free)
Workspace	export_notebook, list_workspace	create_notebook
Unity Catalog	list_tables, get_table_grants, get_table_lineage	—
Compute/DLT	list_clusters, list_pipelines	restart_cluster, start_pipeline_update

Safety model

• Dry-run by default: every write tool returns a validated dry-run message until DBX_WRITE_ENABLED=true. Promote per environment (DEV=true, PROD=false + approval flow).
• Audit log: every tool call appends a JSONL record (timestamp, tool, payload).
• Context safety: results capped at DBX_MAX_RESULT_ROWS / DBX_MAX_RESULT_CHARS.
• Read-only SQL enforcement: statements must start with SELECT/SHOW/DESCRIBE/EXPLAIN/WITH unless writes are enabled; multi-statement submissions are rejected.
• Placeholders only: no real hostnames, principals, or credentials anywhere in code. Everything comes from environment variables.

Setup

pip install -r requirements.txt
cp .env.example .env      # fill in host, token (PAT or OAuth M2M), warehouse ID

Run

# Local (stdio) — for Claude Desktop / Claude Code
python server.py

# Remote (streamable HTTP) — for Databricks Apps / ECS / Lambda-backed deployments
MCP_TRANSPORT=http python server.py

Claude Desktop config

{
  "mcpServers": {
    "databricks_ops": {
      "command": "python",
      "args": ["/path/to/databricks_ops_mcp/server.py"],
      "env": {
        "DATABRICKS_HOST": "https://workspace_placeholder.cloud.databricks.com",
        "DATABRICKS_TOKEN": "token_placeholder",
        "DATABRICKS_WAREHOUSE_ID": "warehouse_id_placeholder",
        "DBX_WRITE_ENABLED": "false"
      }
    }
  }
}

Claude Code

claude mcp add databricks_ops -- python /path/to/databricks_ops_mcp/server.py

Bedrock / custom agents

Run with MCP_TRANSPORT=http behind your FastAPI gateway or as a Databricks App; point the agent's MCP client at the streamable HTTP endpoint. For Databricks Apps hosting, replace the bearer token with app OAuth (on-behalf-of user) so Unity Catalog enforces per-user permissions.

Example flows

• "List failed runs from today, get the status of the worst one, and repair it" → list_failed_runs → get_run_status → repair_run (approval-gated in PROD)
• "Create a notebook that deduplicates the claims feed and preview the target table" → create_notebook → preview_table
• "What feeds this table and who can modify it?" → get_table_lineage + get_table_grants (Conversational Data Steward core)

Extension roadmap

1. create_job_from_spec — NL → metadata-driven pipeline config → Jobs API
2. Column-level lineage + cross-platform trace into MSTR cubes (second MCP server)
3. deploy_bundle via Asset Bundles for Git-first production deployment
4. Approval elicitation (ctx.elicit) on write tools instead of a global flag
5. Cost tools over /api/2.0/usage (billing usage export)

Project layout

databricks_ops_mcp/
├── server.py            # FastMCP/MCPServer registration (SDK 1.x & 2.x compatible)
├── client.py            # Shared async REST client, errors, audit, truncation
├── config.py            # Env-driven settings and guardrails
├── tools/
│   ├── jobs.py          # /api/2.2/jobs/*
│   ├── sql.py           # /api/2.0/sql/statements (submit → poll → fetch)
│   ├── workspace.py     # /api/2.0/workspace/* (notebook import/export)
│   ├── unity_catalog.py # /api/2.1/unity-catalog/* + lineage-tracking
│   └── compute.py       # /api/2.1/clusters/*, /api/2.0/pipelines/*
├── requirements.txt
└── .env.example