Databricks MCP Server

A comprehensive Model Context Protocol (MCP) server that enables AI assistants to interact with Databricks workspaces programmatically. This server provides tools for cluster management, notebook operations, job orchestration, Unity Catalog data governance, user management, permissions control, and FinOps cost analytics.

Features

🖥️ Cluster Management

• List, create, start, stop, and terminate clusters
• Edit cluster configurations and resize clusters
• Get cluster events and logs
• Pin/unpin clusters for organization

📔 Notebook Management

• List, create, read, and update notebooks
• Execute notebooks with parameters
• Import/export notebooks in multiple formats
• Move and organize notebooks in workspace

⚙️ Job Management

• Create and manage jobs with complex workflows
• Schedule jobs with cron expressions
• Trigger manual job runs
• Monitor job execution and retrieve outputs
• Configure notifications and alerts

👥 User & Group Management

• List and manage workspace users
• Create and manage groups
• Add/remove users from groups
• Manage service principals

🗄️ Unity Catalog

• Manage catalogs, schemas, and tables
• Query table metadata and statistics
• List volumes and external locations
• Execute SQL queries on catalog tables

🔒 Permissions & Access Control

• Get and set permissions on workspace objects
• Manage Unity Catalog grants and privileges
• View effective permissions
• Audit access control

💰 FinOps & Cost Analytics

• Analyze cluster costs and DBU consumption
• Track spending trends over time
• Forecast future costs
• Get cost optimization recommendations
• Analyze costs by custom tags

Installation

Prerequisites

• Node.js 18 or higher
• npm or yarn
• A Databricks workspace
• Databricks personal access token or service principal credentials

Install from npm

npm install -g databricks-mcp-server

Install from source

git clone https://github.com/yourusername/databricks-mcp-server.git
cd databricks-mcp-server
npm install
npm run build

Configuration

Environment Variables

Create a .env file in your project directory:

DATABRICKS_HOST=https://your-workspace.cloud.databricks.com
DATABRICKS_TOKEN=your-personal-access-token

For more configuration options, see CONFIGURATION.md.

Claude Desktop Configuration

Add this to your Claude Desktop configuration file:

MacOS: ~/Library/Application Support/Claude/claude_desktop_config.json Windows: %APPDATA%\Claude\claude_desktop_config.json

{
  "mcpServers": {
    "databricks": {
      "command": "databricks-mcp-server",
      "env": {
        "DATABRICKS_HOST": "https://your-workspace.cloud.databricks.com",
        "DATABRICKS_TOKEN": "your-personal-access-token"
      }
    }
  }
}

Quick Start

Once configured, you can interact with your Databricks workspace through your AI assistant:

Example Queries

Cluster Management:

• "List all running clusters in my workspace"
• "Create a new cluster with 4 workers using the latest Spark version"
• "Terminate the idle clusters to save costs"

Notebook Operations:

• "Show me all notebooks in the /Users/me/projects directory"
• "Create a new Python notebook for data analysis"
• "Run the ETL notebook with these parameters"

Job Management:

• "List all scheduled jobs"
• "Create a daily job to run my data pipeline notebook"
• "What's the status of job run 12345?"

Unity Catalog:

• "List all catalogs in my metastore"
• "Show me the schema for the sales.customers table"
• "Query the last 10 records from production.analytics.metrics"

Cost Analytics:

• "What are my cluster costs for the last 30 days?"
• "Show me spending trends by week"
• "Give me cost optimization recommendations"
• "Forecast costs for the next month"

Available Tools

The server provides 60+ tools organized into 7 categories:

Cluster Tools (11 tools)

• list_clusters - List all clusters
• get_cluster_details - Get detailed cluster info
• create_cluster - Create a new cluster
• start_cluster - Start a terminated cluster
• restart_cluster - Restart a cluster
• terminate_cluster - Terminate a cluster
• edit_cluster - Edit cluster configuration
• resize_cluster - Resize cluster workers
• get_cluster_events - Get cluster event history
• pin_cluster - Pin a cluster
• unpin_cluster - Unpin a cluster

Notebook Tools (8 tools)

• list_notebooks - List notebooks in a path
• read_notebook - Read notebook content
• create_notebook - Create a new notebook
• update_notebook - Update notebook content
• delete_notebook - Delete a notebook
• move_notebook - Move/rename a notebook
• create_directory - Create workspace directory
• run_notebook - Execute a notebook

Job Tools (10 tools)

• list_jobs - List all jobs
• get_job_details - Get job configuration
• create_job - Create a new job
• update_job - Update job settings
• delete_job - Delete a job
• run_job - Trigger a job run
• get_job_run_status - Check run status
• list_job_runs - List job run history
• cancel_job_run - Cancel a running job
• get_job_run_output - Get job output

User Tools (11 tools)

• list_users - List all users
• get_user_details - Get user information
• create_user - Add a new user
• delete_user - Remove a user
• list_groups - List all groups
• get_group_details - Get group info
• create_group - Create a new group
• delete_group - Delete a group
• add_user_to_group - Add user to group
• remove_user_from_group - Remove user from group
• list_service_principals - List service principals

Unity Catalog Tools (12 tools)

• list_catalogs - List all catalogs
• get_catalog_details - Get catalog info
• create_catalog - Create a catalog
• delete_catalog - Delete a catalog
• list_schemas - List schemas in catalog
• get_schema_details - Get schema info
• create_schema - Create a schema
• delete_schema - Delete a schema
• list_tables - List tables in schema
• get_table_details - Get table metadata
• delete_table - Delete a table
• list_volumes - List volumes
• list_external_locations - List external locations
• query_table - Execute SQL query

Permission Tools (7 tools)

• get_permissions - Get object permissions
• set_permissions - Set object permissions
• update_permissions - Update object permissions
• get_permission_levels - Get available permission levels
• grant_unity_catalog_permissions - Grant UC privileges
• revoke_unity_catalog_permissions - Revoke UC privileges
• get_effective_permissions - Get effective UC permissions

FinOps Tools (5 tools)

• get_cluster_costs - Analyze cluster costs
• analyze_spending_trends - View spending trends
• forecast_costs - Forecast future costs
• get_optimization_recommendations - Get cost-saving recommendations
• analyze_cost_by_tag - Analyze costs by tags

For detailed documentation of all tools, see API.md.

Development

Building

npm run build

Running in Development

npm run dev

Project Structure

databricks-mcp-server/
├── src/
│   ├── index.ts                 # MCP server entry point
│   ├── databricks-client.ts     # Databricks API client
│   ├── tools/
│   │   ├── clusters.ts          # Cluster management tools
│   │   ├── notebooks.ts         # Notebook management tools
│   │   ├── jobs.ts              # Job management tools
│   │   ├── users.ts             # User/group management tools
│   │   ├── unity-catalog.ts     # Unity Catalog tools
│   │   ├── permissions.ts       # Permission tools
│   │   └── finops.ts            # FinOps tools
│   └── types/
│       └── databricks.ts        # TypeScript type definitions
├── dist/                        # Compiled output
├── package.json
├── tsconfig.json
└── README.md

Examples

See EXAMPLES.md for detailed usage examples and common workflows.

Authentication

The server supports multiple authentication methods:

1. Personal Access Token (recommended for individual use)
2. Service Principal OAuth (recommended for production)
3. Azure AD Token (for Azure Databricks)

See CONFIGURATION.md for detailed authentication setup.

Security Best Practices

• Store credentials in environment variables, never in code
• Use service principals for production deployments
• Implement least-privilege access for service accounts
• Rotate tokens regularly
• Use Unity Catalog for fine-grained data access control

Limitations

• Cost estimates in FinOps tools are approximations. For accurate billing, use the Databricks billing API or account console
• Some operations require specific Databricks editions (Unity Catalog requires Premium/Enterprise)
• Rate limiting is handled automatically with exponential backoff
• SQL execution requires a running SQL warehouse

Troubleshooting

Common Issues

Authentication Failed

• Verify your DATABRICKS_HOST is correct and includes https://
• Check that your DATABRICKS_TOKEN is valid and not expired
• Ensure your token has sufficient permissions

Tool Not Found

• Make sure the server is properly installed and configured
• Check Claude Desktop configuration file syntax
• Restart Claude Desktop after configuration changes

Permission Denied

• Verify your token has the required permissions
• Check workspace access controls
• Review Unity Catalog grants for data access

For more troubleshooting help, see CONFIGURATION.md.

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

MIT License - see LICENSE file for details

Support

For issues, questions, or contributions:

• GitHub Issues: Report an issue
• Documentation: Full API Docs
• Examples: Usage Examples

Related Projects

• Model Context Protocol
• Databricks SDK
• Claude Desktop

Acknowledgments

Built with the Model Context Protocol SDK by Anthropic.