mcp-meeting-analyzer

MCP server for analysing recorded meetings, optimised for efficient context delivery to an LLM.

Problem

Recorded meetings are long and sending the full video to an LLM is inefficient. Most of the time frames contain only people, with no additional informational value beyond what is already in the audio.

Solution

A three-stage processing pipeline:

1. Transcription with timestamps

• Extracts audio from the video
• Transcribes with faster-whisper (local, open-source, 3–4× faster than the original Whisper)
• Produces a segmented transcript with precise timestamps
• Optional diarisation (speaker identification) via pyannote.audio

2. Frame extraction and classification

• Extracts frames from the video with opencv-python
• Detects and discards frames containing people via mediapipe (face/person detection)
• Keeps frames with screens, slides or presentations
• Optimisation: only processes frames with significant change relative to the previous one

3. Temporal association

• Associates each relevant frame with the corresponding transcript segment
• Produces a structured document combining text and images

Output

[00:05:32] "...the decision was to go with option B..."
[FRAME: slide comparing option A vs B]

[00:12:10] "...the deadline is the 20th..."
[FRAME: timeline on screen]

This compact output is sent to the LLM to extract a summary, decisions and action items.

MCP Tools

Main pipeline

Tool	Mode	Description
process_meeting	synchronous	Full pipeline — transcription + frames + temporal association
start_process_meeting	asynchronous	Launches the pipeline in the background; returns job_id immediately
get_process_meeting_status	—	Polls pipeline status; returns result when complete

Individual tools

Tool	Mode	Description
transcribe	synchronous	Transcription only, no frames
start_transcription	asynchronous	Transcription in background; returns job_id immediately
get_transcription_status	—	Polls transcription status
extract_frames	synchronous	Frame extraction only, no transcription

Session management

Tool	Description
get_frame	Retrieves a specific frame as a base64 JPEG
close_session	Removes session from the registry and deletes files from disk

Tech Stack

• MCP SDK: mcp (Python, official Anthropic)
• Transcription: faster-whisper
• Diarisation: pyannote.audio (optional)
• Video/frames: opencv-python (cv2)
• Person detection: mediapipe
• Text/slide detection: opencv-python + easyocr

System Requirements

Binaries that must be installed before using the MCP:

Binary	Purpose	Installation (Ubuntu/Debian)
ffmpeg	Audio extraction from video (used by faster-whisper via PyAV)	sudo apt install ffmpeg
libgles2	Required by mediapipe for face detection	sudo apt install libgles2
python3.12+	Runtime	sudo apt install python3.12
uv	Python package/environment manager	curl -LsSf https://astral.sh/uv/install.sh | sh

The MCP checks these requirements at startup and reports any that are missing.

Installation

git clone https://github.com/mourabraz/mcp-meeting-analyzer
cd mcp-meeting-analyzer
uv sync

Download the mediapipe face detection model:

mkdir -p models
curl -L -o models/blaze_face_short_range.tflite \
  https://storage.googleapis.com/mediapipe-models/face_detector/blaze_face_short_range/float16/1/blaze_face_short_range.tflite

Claude Desktop Configuration

Add to your claude_desktop_config.json:

{
  "mcpServers": {
    "meeting-analyzer": {
      "command": "uv",
      "args": [
        "run",
        "--directory",
        "/path/to/mcp-meeting-analyzer",
        "python",
        "main.py"
      ]
    }
  }
}

Project Structure

mcp-meeting-analyzer/
├── meeting_analyzer/          # main package
│   ├── core/                  # pure domain logic — no MCP, no I/O side effects
│   │   ├── transcriber.py     # transcribe_video() — audio extraction + Whisper
│   │   ├── frame_extractor.py # extract_frames() — capture + classify + dedup pipeline
│   │   ├── frame_classifier.py# classify_frame() — screen vs people via mediapipe
│   │   ├── frame_dedup.py     # deduplicate_frames(), compute_ssim() — SSIM-based dedup
│   │   └── pipeline.py        # process_meeting() — full pipeline orchestration
│   ├── tools/                 # MCP tool handlers (register(mcp) pattern)
│   │   ├── pipeline.py        # process_meeting, start_process_meeting, get_process_meeting_status
│   │   ├── transcription.py   # transcribe, start_transcription, get_transcription_status
│   │   ├── frames.py          # extract_frames, get_frame
│   │   └── session.py         # close_session
│   ├── infra/                 # infrastructure concerns
│   │   ├── checks.py          # startup dependency checks (ffmpeg, libgles2, …)
│   │   ├── debug.py           # MCP_DEBUG=1 structured logging
│   │   └── sessions.py        # in-memory session registry + disk persistence
│   └── server.py              # FastMCP server entry point
├── tests/
│   ├── unit/                  # pytest unit tests (fast, no video files needed)
│   │   ├── conftest.py        # shared fixtures (synthetic numpy frames)
│   │   ├── test_pipeline.py   # _associate_frames_to_segments, _split_inline_on_demand
│   │   ├── test_frame_dedup.py# compute_ssim, deduplicate_frames
│   │   └── test_frame_classifier.py # classify_frame (mocked detector), _compute_content_score
│   ├── test_transcribe.py     # manual integration script — run with uv run python
│   ├── test_e2e_frames.py     # manual integration script — frame extraction smoke test
│   └── test_dedup.py          # manual integration script — deduplication smoke test
├── models/
│   └── blaze_face_short_range.tflite  # mediapipe model (gitignored, download separately)
├── examples/                  # sample video files for manual testing
└── pyproject.toml

Debug Mode

Set MCP_DEBUG=1 to enable structured logging of all tool calls to debug-logs/.

Status

• [x] Project structure
• [x] Python environment setup
• [x] Tool: transcription (sync and async)
• [x] Tool: frame extraction
• [x] Tool: full pipeline (sync and async)
• [x] Tool: get_frame (on-demand retrieval)
• [x] Tool: close_session (session cleanup)
• [x] Persistent sessions (survive server restart)

Tests

There are two kinds of tests: unit tests (pytest, fast) and manual integration scripts (run directly with uv run python).

Unit tests

# Run all unit tests
uv run pytest tests/unit/

# Verbose output — shows each test name
uv run pytest tests/unit/ -v

# Filter by name
uv run pytest tests/unit/ -k "test_ssim"

# Short traceback on failures
uv run pytest tests/unit/ --tb=short

The unit tests cover core/ only. They use synthetic numpy arrays instead of real video files, so they run in under 2 seconds and require no external files. The face detector (_create_face_detector) is automatically skipped if the .tflite model is absent.

Test file	Module under test	Strategy
test_pipeline.py	core/pipeline.py	Pure functions — no I/O, no mocking
test_frame_dedup.py	core/frame_dedup.py	compute_ssim with numpy arrays; deduplicate_frames with tmp_path
test_frame_classifier.py	core/frame_classifier.py	MagicMock to replace the mediapipe detector

Manual integration scripts

These scripts require real video files and a working environment (model downloaded, ffmpeg installed).

# Transcription smoke test
uv run python tests/test_transcribe.py examples/electrao_part1.mp4 pt small

# Frame extraction smoke test (writes frames to output/)
uv run python tests/test_e2e_frames.py

# Deduplication smoke test (requires frames already extracted under output/frames_classified/)
uv run python tests/test_dedup.py

Linting

uv run ruff check .

Contributing

Contributions are welcome. Please open an issue first to discuss what you would like to change.

When submitting a pull request:

• Keep changes focused — one concern per PR
• Follow the existing code style (enforced by ruff)
• Run uv run ruff check . before submitting

Development

Co-authored with Claude Code.
Every design decision was a conversation.

License

MIT