videolingo

VideoLingo is a comprehensive video translation and dubbing tool designed to produce Netflix-quality subtitles. It addresses common issues with machine translation by providing single-line subtitles only, eliminating stiff translations, and adding high-quality dubbing. Key capabilities include: - YouTube video download via yt-dlp - Word-level subtitle recognition with WhisperX (large-v3), runnable locally or via API - NLP and AI-powered subtitle segmentation - Custom and AI-generated terminology for coherent translation - A 3-step Translate-Reflect-Adaptation process for cinematic quality translation - Netflix-standard single-line subtitle output - Dubbing support via GPT-SoVITS, Azure, OpenAI, Fish-TTS, Edge-TTS, and custom TTS options - One-click startup and processing through a Streamlit web UI with multi-language support - Task control allowing users to pause, resume, or stop processing at any step - Detailed logging with progress resumption - A model search box with API auto-fetch to filter models from a provider's full list Supported input languages include English, Russian, French, German, Italian, Spanish, Japanese, and Chinese. Translation supports all languages, while dubbing language depends on the chosen TTS method. Installation options include using uv (recommended) for automated Python environment setup, Conda, or Docker. The project supports OpenAI-compatible API formats for LLM integration and various TTS interfaces. It can run entirely locally with Ollama and Edge-TTS without requiring paid APIs. Current limitations include potential WhisperX transcription issues with background noise, errors from weaker LLMs with strict JSON format requirements, imperfect dubbing due to speech rate differences, retention of only the main language in multilingual videos, and inability to dub multiple characters separately.