whisper

Whisper is a Transformer sequence-to-sequence model trained on a large-scale dataset of diverse audio. It serves as a multitasking system that can replace multiple stages of a traditional speech-processing pipeline by jointly representing various tasks as a sequence of tokens. Key capabilities include: - Multilingual speech recognition (transcription) - Speech translation into English - Spoken language identification - Voice activity detection The project provides six model sizes (tiny, base, small, medium, large, and turbo) to allow users to balance inference speed and accuracy based on their available VRAM. English-only versions are available for the smaller model sizes. Whisper can be used via a command-line interface or as a Python library. It requires ffmpeg for audio processing and utilizes OpenAI's tiktoken for fast tokenization.