Data Science Agent

Autonomous AI agent for data analysis — custom ReAct loop, sandboxed Python/SQL execution, hypothesis testing, MCP protocol, skill system, real-time streaming WebUI.

Features

Core Agent

• Custom ReAct Loop — No LangChain dependency. Full control over planning, tool dispatch, and error recovery
• Dynamic Tool Registry — Pluggable tools with JSON Schema definitions. Add new tools in one line
• Planner — Decomposes vague requests into structured analysis plans
• Streaming WebSocket — Real-time thought → tool → result flow

Analysis Capabilities

• Python Sandbox — Isolated subprocess execution with auto-imported pandas/numpy/matplotlib/seaborn/scipy
• SQL Executor — Auto-loads uploaded files into SQLite tables for JOIN/GROUP BY queries
• Hypothesis Testing Engine — 8 statistical tests (t-test, chi-square, ANOVA, Mann-Whitney U, Pearson/Spearman) with effect sizes (Cohen's d, η², Cramer's V)
• Multi-file Analysis — Cross-table JOIN and correlation analysis across multiple uploaded files
• Chart Generation — Auto-captured matplotlib/seaborn charts displayed inline
• Data Lineage Tracking — Trace any conclusion back to its source data and tool calls

Skill System

• 5 Pre-built Analysis Templates: Financial Analysis, User Segmentation, Anomaly Detection, Correlation Analysis, Time Series Analysis
• Multi-select — Combine multiple skills for comprehensive analysis
• Extensible — Add custom skills as JSON files in skills/

MCP Protocol

• JSON-RPC 2.0 endpoint at /mcp — tools/list, tools/call, resources/list
• Compatible with any MCP client
• 6 tools exposed: file_reader, python_executor, sql_executor, hypothesis_test, skill_loader, finish

Memory & Persistence

• SQLite-backed sessions — Survives restarts and page refreshes
• Auto-titling — Sessions named after uploaded files
• Cleanup on delete — Removing a session deletes its files, reports, and database

Export

• Markdown / HTML / PDF / Jupyter Notebook (.ipynb) — full analysis pipeline as executable notebook
• Streaming chat UI with real-time step visualization
• Configurable LLM — Set API key/base URL/model via UI settings panel

Architecture

User → WebUI (Next.js) → FastAPI
                          ├── ReAct Agent Core
                          ├── Tool Registry (6 tools)
                          ├── MCP Server (/mcp)
                          ├── Skill Registry (5 templates)
                          ├── Python Sandbox (subprocess)
                          ├── SQL Executor (SQLite per session)
                          └── Memory (SQLite persistence)

Quick Start

Prerequisites

• Python 3.9+
• Node.js 22+
• LLM API key (OpenAI, DeepSeek, OpenRouter, etc.)

Setup

git clone <repo-url>
cd data-science-agent

# Backend
cp .env.example .env
# Edit .env with your API key
pip install -r requirements.txt
python -m uvicorn server.main:app --host 0.0.0.0 --port 8000

# Frontend (new terminal)
cd web
npm install
npm run dev

Open http://localhost:3000.

Usage Guide

1. 配置 API 点击右上角齿轮图标 → 填入 DeepSeek / OpenAI / OpenRouter 的 API Key、Base URL、Model → 保存

2. 上传数据 左侧「数据文件」区域拖拽或点击上传 CSV/Excel/JSON 文件。支持多个文件做关联分析。

3. 选择技能（可选） 聊天区顶部选择预置技能模板（财务分析、异常检测、相关性分析等），可多选组合。

4. 提问分析 输入框输入问题，发送。Agent 自动：探查数据 → 写代码 → 出图表 → 统计检验 → 生成报告

5. 查看结果

• 分析步骤和图表实时展示
• 完成后点「查看报告」看完整报告
• 点「数据血缘」追溯数据来源
• 下载 PDF / Markdown / Jupyter Notebook

Docker

docker compose up --build

CLI Usage

# Single analysis
python cli.py "Analyze sales trends by region" -f data.csv -o report.md

# Interactive mode
python cli.py -i

API

Method	Endpoint	Description
POST	/api/sessions	Create session
POST	/api/sessions/{id}/upload	Upload file
POST	/api/sessions/{id}/chat	Send message
WS	/ws/{id}	Streaming chat
GET	/api/sessions/{id}/report	Download report (md/html/pdf)
POST	/mcp	MCP JSON-RPC endpoint
GET	/api/skills	List analysis skills

Project Structure

├── agent/
│   ├── core/          # ReAct loop, planner
│   ├── llm/           # LLM client (OpenAI-compatible)
│   ├── tools/builtin/ # file_reader, python_executor, sql_executor,
│   │                  # hypothesis_test, skill_loader, finish
│   ├── memory/        # Session context + SQLite store
│   ├── sandbox/       # Subprocess & Docker sandbox
│   ├── mcp/           # MCP protocol server
│   ├── skills/        # Skill registry
│   └── reporter/      # Markdown report generator
├── server/            # FastAPI app
├── web/               # Next.js frontend
├── skills/            # Skill template JSON files
├── cli.py             # CLI entry point
└── docker-compose.yml

Key Design Decisions

1. No LangChain — Full control over agent loop and tool dispatch
2. MCP-native — Tools exposed via standard protocol, not proprietary API
3. Hypothesis-driven — Beyond descriptive stats; every conclusion backed by statistical tests
4. Lineage tracking — Every data point traceable to its source
5. Skill templates — Reusable, validated analysis workflows

License

MIT