VibeVoice TTS Server

Local OpenAI-compatible text-to-speech API server powered by VibeVoice-7B. Generates up to 45 minutes of speech in a single request, with voice cloning from reference audio.

Features

• OpenAI-compatible API — drop-in replacement for POST /v1/audio/speech
• Voice cloning — upload reference audio to clone any speaker's voice
• Long-form generation — natively supports up to ~45 minutes per request (full podcast episodes)
• On-demand model loading — loads the 7B model on first request, auto-unloads after idle timeout to free VRAM
• Multiple formats — MP3, WAV, OPUS, FLAC, AAC, PCM
• MCP server — use as a Claude Code tool for speech synthesis
• Platform detection — CUDA > MPS > CPU with optional 4-bit quantization

Quickstart

1. Install

pip install -e .

# With CUDA flash attention:
pip install -e ".[cuda]"

# With 4-bit quantization:
pip install -e ".[quant]"

Requires ffmpeg for MP3/OPUS/AAC encoding:

# macOS
brew install ffmpeg

# Ubuntu/Debian
sudo apt install ffmpeg

2. Start the server

vibevoice-tts-server

The model downloads on first request (~14 GB) and loads into VRAM/RAM. Subsequent requests reuse the loaded model.

Options:

vibevoice-tts-server --device cuda --port 8100 --idle-timeout 600

3. Generate speech

curl -X POST http://localhost:8101/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{"input": "Hello world, this is VibeVoice!", "voice": "alloy"}' \
  --output hello.mp3

4. Voice cloning with reference audio

Upload a reference audio file to clone a speaker's voice:

curl -X POST http://localhost:8101/v1/audio/speech/upload \
  -F "input=Welcome to the show, I'm your host." \
  -F "voice=alloy" \
  -F "response_format=wav" \
  -F "reference_audio=@speaker_sample.wav" \
  --output cloned.wav

5. Multi-speaker generation (one-shot)

VibeVoice natively generates multi-speaker audio in a single pass. Format the input with Speaker N: prefixes and upload one reference audio file per speaker:

curl -X POST http://localhost:8101/v1/audio/speech/upload \
  -F "input=Speaker 1: Welcome to the show, I'm your host.
Speaker 2: Thanks for having me, great to be here.
Speaker 1: Let's dive right in." \
  -F "response_format=wav" \
  -F "reference_audio=@host_voice.wav" \
  -F "reference_audio=@guest_voice.wav" \
  --output podcast.wav

Or via the JSON endpoint with file paths:

curl -X POST http://localhost:8101/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Speaker 1: Welcome to the show.\nSpeaker 2: Thanks for having me.",
    "instructions": "{\"reference_audio\": [\"/path/to/host.wav\", \"/path/to/guest.wav\"]}"
  }' \
  --output podcast.mp3

Single-speaker voice cloning also works with a single file path:

curl -X POST http://localhost:8101/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "input": "Welcome to the show.",
    "voice": "alloy",
    "instructions": "{\"reference_audio\": \"/path/to/speaker_sample.wav\"}"
  }' \
  --output cloned.mp3

API Reference

POST /v1/audio/speech

JSON body (OpenAI-compatible):

Field	Type	Default	Description
input	string	required	Text to synthesize
voice	string	"alloy"	Voice preset or speaker name
model	string	"vibevoice-tts"	Model identifier
response_format	string	"mp3"	mp3, wav, opus, flac, aac, pcm
speed	float	1.0	Speed multiplier (0.25 - 4.0)
instructions	string	null	JSON string with advanced params (see below)

Instructions JSON fields:

Field	Type	Description
reference_audio	string or string[]	Path(s) to reference audio — single path for one speaker, array for multi-speaker
cfg_scale	float	Classifier-free guidance scale (default: 1.3)
n_diffusion_steps	int	Diffusion denoising steps (default: 10)
max_new_tokens	int	Audio token limit at 7.5 Hz (0 = unlimited)

Returns raw audio bytes with the appropriate Content-Type header.

POST /v1/audio/speech/upload

Multipart form — same fields as above, plus:

Field	Type	Description
reference_audio	file(s)	One or more audio files for voice cloning — upload multiple for multi-speaker (ordered by Speaker 1, 2, etc.)

GET /v1/audio/voices

List available voice presets.

GET /v1/models

List available models.

GET /health

Server status, model load state, device info.

Voice Presets

OpenAI Name	VibeVoice Speaker
alloy	Emma
echo	Carter
fable	Davis
onyx	Mike
nova	Grace
shimmer	Frank
sage	Samuel

Configuration

All settings can be set via environment variables with the VIBEVOICE_TTS_ prefix:

VIBEVOICE_TTS_HOST=0.0.0.0
VIBEVOICE_TTS_PORT=8101
VIBEVOICE_TTS_MODEL_ID=vibevoice/VibeVoice-7B
VIBEVOICE_TTS_DEVICE=auto          # auto, cuda, mps, cpu
VIBEVOICE_TTS_DTYPE=auto           # auto, bfloat16, float32
VIBEVOICE_TTS_IDLE_TIMEOUT=300     # seconds before unloading model (0 = never)
VIBEVOICE_TTS_MAX_NEW_TOKENS=0     # 0 = unlimited; tokens are audio frames at 7.5 Hz
VIBEVOICE_TTS_CFG_SCALE=1.3
VIBEVOICE_TTS_N_DIFFUSION_STEPS=10
VIBEVOICE_TTS_QUANTIZE_4BIT=false

MCP Server

Use as a Claude Code tool:

vibevoice-tts-mcp

Add to your Claude Code MCP config:

{
  "mcpServers": {
    "vibevoice-tts": {
      "command": "vibevoice-tts-mcp",
      "args": ["--device", "auto"]
    }
  }
}

Tools: synthesize_speech, list_voices, get_tts_status

Running Tests

pip install -e ".[test]"
pytest tests/ -v

Audio Token Math

VibeVoice generates audio tokens at 7.5 Hz (7.5 tokens per second of audio). The 7B model has a 32K context window shared between text input tokens and audio output tokens.

Tokens	Duration
450	~1 minute
4,500	~10 minutes
13,500	~30 minutes
20,250	~45 minutes

By default, max_new_tokens=0 (unlimited), allowing the model to generate until it finishes the input text naturally.

Licensing

This server code is released under the MIT License.

Model license note: Microsoft released VibeVoice-7B under the MIT License. However, Microsoft's model card states the model is "limited to research purpose use" and later removed the TTS code from their official repository citing misuse concerns. Community forks and model weights remain available under MIT. Users should review the model card and applicable terms before deploying in production.

This project is an independent wrapper and is not affiliated with or endorsed by Microsoft.