DLT, FastAPI, and FastMCP Integration

This project is a demonstration of how to integrate dlt (data load tool), FastAPI, and FastMCP to create a simple cryptocurrency market analysis agent.

Project Overview

The project consists of three main components:

1. Data Pipeline: A dlt pipeline that ingests cryptocurrency market data from the CoinGecko API and loads it into a local DuckDB database.
2. API Backend: A FastAPI application that exposes the data from the DuckDB database through a REST API.
3. Agent: A FastMCP agent that provides tools to interact with the API, refresh the data, and perform market analysis.

How it Works

┌───────────────────┐      ┌──────────────────┐      ┌──────────────────┐
│   CoinGecko API   │ ◄─── │   dlt Pipeline   │ ───► │  DuckDB Database │
└───────────────────┘      └──────────────────┘      └──────────────────┘
                                                          ▲
                                                          │
                                                          │
┌───────────────────┐      ┌──────────────────┐      ┌────┴─────┐
│      User         │ ◄─── │  FastMCP Agent   │ ◄─── │  FastAPI  │
└───────────────────┘      └──────────────────┘      └──────────┘

Components

Data Pipeline (dlt)

The data pipeline is defined in data_pipeline/ingest_coins.py. It uses the dlt library to:

1. Fetch the top 10 cryptocurrencies by market cap from the CoinGecko API.
2. Load the data into a DuckDB database named crypto_pipeline.duckdb.
3. The data is stored in a table named top_coins within the market_data schema.

API Backend (FastAPI)

The API backend is defined in backend.py. It uses the FastAPI framework to create a simple API with the following endpoints:

• GET /: Returns a status message.
• GET /coins: Returns a list of cryptocurrencies from the database. It supports min_price and limit query parameters for filtering and pagination.

Agent (fastmcp)

The agent is defined in agent.py. It uses the FastMCP framework to create an agent with the following tools:

• get_crypto_market_data: Fetches cryptocurrency data from the FastAPI backend.
• refresh_data: Triggers the dlt pipeline to refresh the data from the CoinGecko API.
• analyze_market: Performs a simple market analysis on the data.

How to Run

1. Install Dependencies

Install the required Python packages from requirements.txt:

pip install -r requirements.txt

2. Run the Data Pipeline

Run the data pipeline to populate the database:

python data_pipeline/ingest_coins.py

This will create a crypto_pipeline.duckdb file in the project root.

3. Run the API Backend

Start the FastAPI server:

python backend.py

The API will be available at http://localhost:8000.

4. Run the Agent

In a separate terminal, run the agent:

python agent.py

You can now interact with the agent in your terminal.

Verifying the Data

You can manually verify the data in the database using the provided scripts:

• check_data.py: Shows the top 5 coins by price.
• check_metadata.py: Shows dlt metadata for the loaded data.

Run them like this:

python check_data.py
python check_metadata.py

Adding the Agent to Gemini CLI

To add the FastMCP agent to the Gemini CLI, create a settings.json file inside your .gemini folder (if it doesn't already exist). Then, copy and paste the following configuration into your settings.json file:

{
  "mcpServers": {
    "Crypto Sentinel Agent": {
      "command": "/Users/adpuz/Documents/Projects/dlt_fastapi_mcp/.venv/bin/python",
      "args": [
        "/Users/adpuz/Documents/Projects/dlt_fastapi_mcp/agent.py"
      ]
    }
  }
}

Make sure the command and args paths are correct for your environment. After saving the settings.json file, you can use the @Crypto Sentinel Agent in the Gemini CLI to interact with your agent, for example, by typing @Crypto Sentinel Agent check the latest crypto market data.