MCP Observability Server

A Model Context Protocol (MCP) server that demonstrates how to incorporate comprehensive observability into MCP servers using Sentry, OpenTelemetry, and Braintrust.

Features

🔧 Tools

• File Operations: Read files, write files, list directories
• Data Processing: JSON manipulation, data transformation
• HTTP Client: URL fetching, webhooks, health checks

📊 Resources

• Server Status: Real-time server metrics and performance data
• Configuration: Server and observability configuration details
• Health Check: Service health monitoring endpoint

📈 Observability Stack

Sentry

• Error tracking and performance monitoring
• Automatic breadcrumb collection
• User context and custom tags
• Transaction performance tracking

OpenTelemetry

• Distributed tracing with Jaeger export
• Custom metrics with Prometheus export
• Automatic instrumentation of Node.js libraries
• Custom spans for tool executions

Braintrust

• AI/ML model evaluation and logging
• Tool execution tracking
• Performance analytics
• Experiment management

Quick Start

1. Installation

npm install

2. Configuration

Copy the example environment file:

cp .env.example .env

Edit .env with your configuration:

# Sentry (optional)
SENTRY_DSN=your_sentry_dsn_here
SENTRY_ENVIRONMENT=development

# Braintrust (optional) 
BRAINTRUST_API_KEY=your_braintrust_api_key_here
BRAINTRUST_PROJECT_NAME=mcp-observability

# OpenTelemetry (configured by default)
OTEL_SERVICE_NAME=mcp-observability-server
OTEL_EXPORTER_JAEGER_ENDPOINT=http://localhost:14268/api/traces

3. Build and Run

# Development mode
npm run dev

# Production mode
npm run build
npm start

Using with Claude Desktop

Add to your Claude Desktop MCP configuration:

{
  "mcpServers": {
    "observability-server": {
      "command": "node",
      "args": ["/path/to/mcp-observability-server/dist/index.js"]
    }
  }
}

Observability Setup

Sentry Setup

1. Create a Sentry project at sentry.io
2. Copy your DSN to the SENTRY_DSN environment variable
3. The server will automatically track errors and performance

OpenTelemetry + Jaeger Setup

1. Run Jaeger locally:

docker run -d --name jaeger \
  -p 16686:16686 \
  -p 14268:14268 \
  jaegertracing/all-in-one:latest

2. Access Jaeger UI at http://localhost:16686
3. Traces will be automatically exported

Braintrust Setup

1. Sign up at braintrust.dev
2. Get your API key and project name
3. Set the environment variables
4. Tool executions will be logged for analysis

Example Usage

Using Tools

// Read a file
{
  "tool": "read_file",
  "arguments": {
    "path": "/path/to/file.txt",
    "encoding": "utf8"
  }
}

// Process JSON data
{
  "tool": "process_json", 
  "arguments": {
    "data": "{\"users\": [{\"name\": \"Alice\"}, {\"name\": \"Bob\"}]}",
    "operation": "extract_keys",
    "path": "users"
  }
}

// Fetch URL
{
  "tool": "fetch_url",
  "arguments": {
    "url": "https://api.github.com/users/octocat",
    "method": "GET"
  }
}

Accessing Resources

// Get server status
{
  "resource": "status://server"
}

// Health check
{
  "resource": "status://health"
}

// Configuration
{
  "resource": "config://server"
}

Architecture

src/
├── observability/          # Observability integrations
│   ├── sentry.ts          # Sentry error tracking
│   ├── opentelemetry.ts   # OpenTelemetry tracing
│   ├── braintrust.ts      # Braintrust logging
│   └── index.ts           # Unified observability wrapper
├── tools/                 # MCP tools
│   ├── fileOperations.ts  # File I/O tools
│   ├── dataProcessing.ts  # Data manipulation tools
│   ├── httpClient.ts      # HTTP client tools
│   └── index.ts           # Tool registry
├── resources/             # MCP resources  
│   ├── status.ts          # Server status resource
│   ├── config.ts          # Configuration resource
│   └── index.ts           # Resource registry
└── index.ts               # Main server entry point

Key Observability Features

1. Automatic Tool Wrapping

All tools are automatically wrapped with the withObservability decorator that:

• Creates OpenTelemetry spans
• Starts Sentry transactions
• Logs to Braintrust
• Records performance metrics

2. Error Handling

• Automatic error capture in Sentry with context
• Error spans in OpenTelemetry traces
• Failed execution logging in Braintrust

3. Performance Monitoring

• Response time tracking
• Memory usage monitoring
• Request success/failure rates
• Custom metrics via OpenTelemetry

4. Distributed Tracing

• End-to-end request tracing
• Tool execution spans
• External API call tracing
• Performance bottleneck identification

Development

# Type checking
npm run typecheck

# Linting  
npm run lint

# Testing
npm test

License

MIT