whisperlivekit

WhisperLiveKit (WLK) is an open-source, self-hosted speech-to-text pipeline designed for ultra-low-latency real-time transcription. It can be installed via pip and provides a command-line interface to start a server, transcribe audio files, generate SRT subtitles, and manage models. The kit integrates multiple streaming speech processing strategies and backends, including: - Whisper-family models via Faster-Whisper, MLX, or vanilla Whisper. - Voxtral Mini for multilingual speech recognition with per-chunk language detection. - FunASR SenseVoiceSmall for specific languages. - Qwen3-ASR for streaming transcription, including a causal mode for constant compute per audio chunk. - NVIDIA Canary-1b-v2 via NeMo for European languages. WLK supports simultaneous translation to and from 200 languages using NLLW or Alignatt4LLM. It also features real-time speaker diarization via Streaming Sortformer or Diart. The server exposes a native WebSocket endpoint for real-time streaming, alongside OpenAI-compatible REST and Deepgram-compatible WebSocket APIs. It supports multiple concurrent users, voice activity detection (VAD) to reduce overhead, and per-session parameters for language, translation target, terminology context, and protocol mode. The repository includes a bundled web UI, a Chrome extension for capturing web audio, and a native SwiftUI macOS client.