espnet

ESPnet is an end-to-end speech processing toolkit built on PyTorch. It covers a wide range of tasks including automatic speech recognition (ASR), text-to-speech (TTS), speech translation (ST), speech enhancement and separation (SE/SS), speaker diarization, spoken language understanding (SLU), singing voice synthesis (SVS), speech language models (SpeechLM), neural audio codecs, and more. The toolkit provides Kaldi-style reproducible recipes that handle everything from data preparation to evaluation, and offers hundreds of pretrained models on Hugging Face. Key features include: - **ASR**: Hybrid CTC/attention, Transducer, streaming, Conformer/E-Branchformer, Whisper, SSL front-ends, and transfer learning. - **TTS**: Tacotron2, FastSpeech2, VITS, JETS, multi-speaker/multilingual support, and integration with neural vocoders. - **ST/MT**: End-to-end and cascaded speech translation, machine translation, and speech-to-speech translation. - **SE/SS**: Unified encoder-separator-decoder, TasNet/DPRNN, beamformers, and ASR-integrated enhancement. - **SLU**: Intent and transcript multitasking, pretrained ASR/NLP encoders. - **Speaker & language**: Speaker embeddings, verification, language ID, diarization. - **SSL**: HuBERT pretraining and S3PRL upstreams. - **SpeechLM**: Unified sequence modeling across speech and text tasks. - **Codec**: Neural audio codecs for discrete speech tokens. ESPnet emphasizes reproducibility with a single `run.sh` per corpus, unified recipe structure and config format, and scalability via DDP, multi-node training, Slurm/MPI, DeepSpeed, and sharded training. It also provides real-time demos, Colab notebooks, and integration with Hugging Face Spaces and Weights & Biases.