📊 Autonomous Data Analysis System (MCP)

An AI data analyst that answers natural-language questions over CSV / Excel / SQL data. The language model plans and interprets — all computation (SQL, stats, cleaning, charts) runs in deterministic MCP tools, so results are exact, reproducible, and auditable.

Core principle: the LLM never does arithmetic. It reads schemas, chooses which tool to call with which arguments, and turns the returned numbers into business insight.

Architecture

┌──────────────────────────────────────────────────────────┐
│  Streamlit UI   (upload · chat · charts · report · switch) │
└───────────────┬──────────────────────────────────────────┘
                │
┌───────────────▼──────────────────────────────────────────┐
│  Agent host                                                │
│   • Provider switch (LiteLLM): Ollama · Groq · Gemini      │
│   • MCP client: MCP tool schemas ⇆ OpenAI function calls   │
│   • Loop: plan → call tool → observe → answer              │
└───────────────┬──────────────────────────────────────────┘
                │ MCP protocol (stdio)
┌───────────────▼──────────────────────────────────────────┐
│  MCP analysis server                                       │
│   load · profile · sql · eda · clean · stats · viz · report│
│   Dataset registry · Pandas · DuckDB · SciPy · matplotlib  │
└──────────────────────────────────────────────────────────┘

See DESIGN.md for the full design and milestone tracker.

Features

• Upload CSV / Excel; DuckDB SQL over your files with no database server.
• Automated profiling & EDA — schema, nulls, cardinality, correlations, group aggregates.
• Non-destructive cleaning — missing values, duplicates, type casts, outliers (each returns a new versioned dataset).
• Statistics — t-test / ANOVA / chi-square, correlation tests, trend analysis, distribution/normality.
• Charts — bar, line, scatter, histogram, box, correlation heatmap (PNG).
• Self-contained HTML reports with embedded charts.
• Pluggable LLM — local Ollama by default, switch to Groq or Gemini free tiers.

Setup

python3 -m venv .venv
source .venv/bin/activate          # fish: source .venv/bin/activate.fish
pip install -r requirements.txt
cp .env.example .env               # then fill in the provider you want

Choosing a provider (edit .env)

Provider	Setup	Notes
Ollama (default, local)	ollama pull qwen2.5:3b and run the Ollama daemon	Free & private; smaller models plan tools less reliably
Groq (free tier)	GROQ_API_KEY=... from https://console.groq.com/keys	Fast, strong tool-calling — best for reliable planning
Gemini (free tier)	GEMINI_API_KEY=... from https://aistudio.google.com/app/apikey	Large context, strong function-calling

You can also switch provider live from the sidebar dropdown in the UI.

Run the app

streamlit run ui/streamlit_app.py

Then upload one of the samples in data/ (e.g. sample_sales.csv) and ask things like "Which region sells the most units, and is the trend rising?" or "Which plan has the highest churn — build me a report."

Use the MCP server directly (e.g. Claude Desktop)

The server speaks stdio and works with any MCP client:

python -m mcp_server.server

Claude Desktop config:

{ "mcpServers": {
    "data-analysis": { "command": "python", "args": ["-m", "mcp_server.server"] }
} }

MCP tool catalog

Group	Tools
Load	load_csv, load_excel, list_datasets
Profile	profile_dataset
SQL	run_sql (DuckDB)
EDA	value_counts, correlations, groupby_aggregate
Clean	drop_duplicates, handle_missing, cast_types, handle_outliers, rename_columns
Stats	hypothesis_test, correlation_test, trend_analysis, distribution_fit
Viz	make_chart
Report	export_report

Testing

PYTHONPATH="$PWD" pytest -q

15 tests spawn the real MCP server over stdio and exercise every tool group; the agent loop is tested with a scripted fake LLM (deterministic, no network).

Project structure

mcp_server/     FastMCP server + registry + tools/
agent/          provider switch (config, providers) + mcp_client + agent loop
ui/             streamlit_app.py
data/           sample datasets + uploads/
reports/        generated charts and HTML reports
tests/          per-milestone test suites