VideoLingo
Netflix-level subtitle cutting, translation, alignment, and even dubbing - one-click fully automated AI video subtitle team | Netflix级字幕切割、翻译、对齐、甚至加上配音,一键全自动视频搬运AI字幕组 - CXL-edu/VideoLingo
CXL-edu
README
<div align="center">
<img src="/docs/logo.png" alt="VideoLingo Logo" height="140">
Connect the World, Frame by Frame
English|简体中文|繁體中文|日本語|Español|Русский|Français
</div>
🌟 Overview (Try VL Now!)
VideoLingo is an all-in-one video translation, localization, and dubbing tool aimed at generating Netflix-quality subtitles. It eliminates stiff machine translations and multi-line subtitles while adding high-quality dubbing, enabling global knowledge sharing across language barriers.
Key features:
-
🎥 YouTube video download via yt-dlp
-
🎙️ Word-level and Low-illusion subtitle recognition with WhisperX
-
📝 NLP and AI-powered subtitle segmentation
-
📚 Custom + AI-generated terminology for coherent translation
-
🔄 3-step Translate-Reflect-Adaptation for cinematic quality
-
✅ Netflix-standard, Single-line subtitles Only
-
🗣️ Dubbing with GPT-SoVITS, Azure, OpenAI, and more
-
🚀 One-click startup and processing in Streamlit
-
🌍 Multi-language support in Streamlit UI
-
📝 Detailed logging with progress resumption
Difference from similar projects: Single-line subtitles only, superior translation quality, seamless dubbing experience
🎥 Demo
<table> <tr> <td width="50%">
Russian Translation
https://github.com/user-attachments/assets/25264b5b-6931-4d39-948c-5a1e4ce42fa7
</td> <td width="50%">
GPT-SoVITS Dubbing
https://github.com/user-attachments/assets/47d965b2-b4ab-4a0b-9d08-b49a7bf3508c
</td> </tr> </table>
Language Support
Input Language Support(more to come):
🇺🇸 English 🤩 | 🇷🇺 Russian 😊 | 🇫🇷 French 🤩 | 🇩🇪 German 🤩 | 🇮🇹 Italian 🤩 | 🇪🇸 Spanish 🤩 | 🇯🇵 Japanese 😐 | 🇨🇳 Chinese* 😊
*Chinese uses a separate punctuation-enhanced whisper model, for now...
Translation supports all languages, while dubbing language depends on the chosen TTS method.
Installation
You don't have to read the whole docs, here is an online AI agent to help you.
Note: For Windows users with NVIDIA GPU, follow these steps before installation:
- Install CUDA Toolkit 12.6
- Install CUDNN 9.3.0
- Add
C:\Program Files\NVIDIA\CUDNN\v9.3\bin\12.6
to your system PATH- Restart your computer
Note: FFmpeg is required. Please install it via package managers:
- Windows:
choco install ffmpeg
(via Chocolatey)- macOS:
brew install ffmpeg
(via Homebrew)- Linux:
sudo apt install ffmpeg
(Debian/Ubuntu)
- Clone the repository
git clone https://github.com/Huanshere/VideoLingo.git
cd VideoLingo
- Install dependencies(requires
python=3.10
)
conda create -n videolingo python=3.10.0 -y
conda activate videolingo
python install.py
- Start the application
streamlit run st.py
Docker
Alternatively, you can use Docker (requires CUDA 12.4 and NVIDIA Driver version >550), see Docker docs:
docker build -t videolingo .
docker run -d -p 8501:8501 --gpus all videolingo
APIs
VideoLingo supports OpenAI-Like API format and various TTS interfaces:
- LLM:
claude-3-5-sonnet-20240620
,deepseek-chat(v3)
,gemini-2.0-flash-exp
,gpt-4o
, ... (sorted by performance) - WhisperX: Run whisperX locally or use 302.ai API
- TTS:
azure-tts
,openai-tts
,siliconflow-fishtts
,fish-tts
,GPT-SoVITS
,edge-tts
,*custom-tts
(You can modify your own TTS in custom_tts.py!)
Note: VideoLingo works with 302.ai - one API key for all services (LLM, WhisperX, TTS). Or run locally with Ollama and Edge-TTS for free, no API needed!
For detailed installation, API configuration, and batch mode instructions, please refer to the documentation: English | 中文
Current Limitations
-
WhisperX transcription performance may be affected by video background noise, as it uses wav2vac model for alignment. For videos with loud background music, please enable Voice Separation Enhancement. Additionally, subtitles ending with numbers or special characters may be truncated early due to wav2vac's inability to map numeric characters (e.g., "1") to their spoken form ("one").
-
Using weaker models can lead to errors during intermediate processes due to strict JSON format requirements for responses. If this error occurs, please delete the
output
folder and retry with a different LLM, otherwise repeated execution will read the previous erroneous response causing the same error. -
The dubbing feature may not be 100% perfect due to differences in speech rates and intonation between languages, as well as the impact of the translation step. However, this project has implemented extensive engineering processing for speech rates to ensure the best possible dubbing results.
-
Multilingual video transcription recognition will only retain the main language. This is because whisperX uses a specialized model for a single language when forcibly aligning word-level subtitles, and will delete unrecognized languages.
-
Cannot dub multiple characters separately, as whisperX's speaker distinction capability is not sufficiently reliable.
📄 License
This project is licensed under the Apache 2.0 License. Special thanks to the following open source projects for their contributions:
whisperX, yt-dlp, json_repair, BELLE
📬 Contact Me
- Submit Issues or Pull Requests on GitHub
- DM me on Twitter: @Huanshere
- Email me at: team@videolingo.io
⭐ Star History
<p align="center">If you find VideoLingo helpful, please give me a ⭐️!</p>
Recommended Servers
Mult Fetch MCP Server
A versatile MCP-compliant web content fetching tool that supports multiple modes (browser/node), formats (HTML/JSON/Markdown/Text), and intelligent proxy detection, with bilingual interface (English/Chinese).
Hyperbrowser MCP Server
Welcome to Hyperbrowser, the Internet for AI. Hyperbrowser is the next-generation platform empowering AI agents and enabling effortless, scalable browser automation. Built specifically for AI developers, it eliminates the headaches of local infrastructure and performance bottlenecks, allowing you to
Web Research Server
A Model Context Protocol server that enables Claude to perform web research by integrating Google search, extracting webpage content, and capturing screenshots.

Youtube Translate
A Model Context Protocol server that enables access to YouTube video content through transcripts, translations, summaries, and subtitle generation in various languages.
Fetch MCP Server
Provides functionality to fetch web content in various formats, including HTML, JSON, plain text, and Markdown.
Web Research Server
MCP web research server (give Claude real-time info from the web) - oneshot-engineering/mcp-webresearch
Jina AI
Contribute to JoeBuildsStuff/mcp-jina-ai development by creating an account on GitHub.

Workflow Learner
To teach LLM Agents understand workflows with PSR.exe's hmt.

Chat Summary Server
summarize chat message
DEEP Web Research
The MCP Web Research Server enables real-time web research with Claude by integrating Google search, capturing webpage content and screenshots, and tracking research sessions.