FinSight Copilot: Financial Research & Analysis RAG Agent

FinSight Copilot is an end-to-end, AI-powered financial research assistant for financial analysts. Built on top of the Anthropic Claude Messages API, PostgreSQL with pgvector, real-time financial market tools and the Model Context Protocol (MCP), FinSight Copilot provides cited earnings report analysis, deterministic portfolio risk metrics, and guardrail-enforced financial research capabilities.

---

Key Capabilities & Features

• Analyst Copilot Engine: Powered by Claude Messages API with XML-structured prompts, Chain-of-Thought (CoT) reasoning, structured output parsing, and real-time response handling.
• Vector RAG with In-line Citations: Grounded analysis over SEC filings (10-Ks, earnings disclosures, and transcripts) using pgvector. Every claim includes strict document and section citations.
• Live Financial Tools: Real-time financial market lookups (quotes, P/E ratios, trading volumes) integrated via external financial data APIs (Yahoo Finance / yfinance).
• Deterministic Portfolio Math: Value at Risk (VaR) calculations combined with enforced constitutional disclaimers.
• Model Context Protocol (MCP): Exposes financial tools as standardized MCP endpoints using fastmcp.
• Optimization & Caching: Leverages active Anthropic Prompt Caching (cache_control={"type": "ephemeral"}) to reduce input token overhead by up to ~90% on repeated RAG queries.
• Evaluation & Defense Harness: Includes an automated test suite verifying grounding accuracy, prompt injection defense, guardrail adherence, and execution cost tracking.

---

Project Structure

finsight-copilot/
│
├── data/
│   ├── 10k_reports/             # Seed Data: Raw PDF filings (e.g., Tesla, Nvidia, Amazon 10-Ks)
│   ├── research_notes/          # Supplementary RAG context (e.g., tech_sector_2024.txt)
│   └── transcripts/             # Adversarial testing data (e.g., malicious_hack.txt for prompt injection defenses)
│
├── src/
│   ├── agent.py                 # Autonomous State Machine loop (e.g., FETCH -> FLAG_RISK -> SUMMARIZE)
│   ├── copilot.py               # Main agent logic, tool routing, guardrails, and prompt caching
│   ├── eval.py                  # Automated evaluation suite testing accuracy, security, and token costs
│   ├── mcp_server.py            # Model Context Protocol (MCP) server exposing tools via fastmcp
│   ├── rag.py                   # Ingestion pipeline, PDF sliding-window chunker, and pgvector engine
│   ├── skills.py                # Modular AI skill definitions and instructions used by the agent/server
│   └── tools.py                 # Live yfinance integration and portfolio risk (VaR) math functions
│
├── .env                         # Local environment variables (Git ignored)
├── .env.example                 # Template for required environment variables
├── requirements.txt             # Python project dependencies
└── README.md                    # Project documentation

---

Prerequisites

Ensure you have the following installed on your machine before setup:

• Python: v3.10 or higher
• Docker Desktop: Required for running the PostgreSQL + pgvector container (or local PostgreSQL v15+ with pgvector)
• Anthropic API Key: Active key from the Anthropic Console

---

Installation & Setup Guide

Step 1: Clone Project Directory

git clone https://github.com/ayadav6/finsight-copilot.git
cd finsight-copilot

Step 2: Set Up Python Virtual Environment (venv)

Create and activate an isolated Python virtual environment:

macOS / Linux:

python3 -m venv venv
source venv/bin/activate

Windows (Command Prompt):

python -m venv venv
venv\Scripts\activate.bat

Windows (PowerShell):

python -m venv venv
.\venv\Scripts\Activate.ps1

Step 3: Install Project Dependencies

pip install --upgrade pip
pip install -r requirements.txt

(Note: fastmcp is included in requirements.txt for the MCP server functionality).

Step 4: Configure Environment Variables

Create a .env file in the root directory:

touch .env

Add your API credentials and database connection details to .env:

# Anthropic API Configuration
ANTHROPIC_API_KEY=your_anthropic_api_key_here

# PostgreSQL database connection string (Default for local Docker pgvector)
DATABASE_URL=postgresql://postgres:postgres@localhost:5432/postgres

Step 5: Start & Configure PostgreSQL Vector Database

Start a containerized PostgreSQL instance with pgvector enabled using Docker:

docker run --name pgvector-finsight \
  -e POSTGRES_USER=postgres \
  -e POSTGRES_PASSWORD=postgres \
  -e POSTGRES_DB=postgres \
  -p 5432:5432 \
  -d pgvector/pgvector:pg16

---

Execution Guide

Step 1: Seed Data & Ingest Financial Disclosures

Place all target SEC Form 10-K PDFs (e.g., Tesla_2025_10K.pdf, Nvidia_2025_10K.pdf, Amazon_2025_10K.pdf) into the data/10k_reports/ directory.

Run the RAG document ingestion script to parse disclosures, chunk text using sliding context windows, compute embeddings, and populate PostgreSQL:

python src/rag.py

(Note: Ingesting multi-page PDF filings locally on CPU can take between 5 to 15 minutes. The script contains an automated document count check to bypass redundant re-ingestion on subsequent executions).

Step 2: Running FinSight Copilot

Once the vector database is populated with financial disclosures, run the main interactive copilot application:

python src/copilot.py

Supported Operations in copilot.py:

• Live Market Lookups: Fetches current stock prices, volume, and P/E ratios (e.g., "What is NVDA's current stock price and P/E ratio?").
• Portfolio Risk Calculations: Computes deterministic Value at Risk (VaR) metrics for user portfolios while appending mandatory disclaimers.
• Document Grounded RAG: Inquires about historical financial data (e.g., "Summarize Tesla's related party transactions with xAI in 2025 and 2026.") with document citations.

Step 3: Running the Automated Evaluation Harness

To run adversarial compliance tests, verify grounded citations, check guardrails, and generate an execution token micro-cost report:

python src/eval.py

Step 4: Running the MCP Server

FinSight's tools are packaged into an MCP server. Run the MCP server inspector using:

fastmcp dev inspector src/mcp_server.py

This command launches a local web server (usually http://localhost:5173) where you can visually inspect, connect, and test the get_stock_quote and compute_portfolio_risk tools directly via a web UI.

---

Engineering & Architectural Highlights

Feature	Description
Prompt Caching	Utilizes cache_control={"type": "ephemeral"} to lower input token overhead from ~4,000 tokens down to 1 token on repeated system prompts, reducing latency and costs by up to 90%.
Context Windowing	Overlapping sliding window text chunking (1500 characters, 300-character overlap) prevents splitting financial figures, tables, or key sentences across boundaries.
Constitutional Guardrails	System instructions enforce strict non-personalized advice policies and mandate liability disclaimers on portfolio risk outputs.