retrieval-based-voice-conversion-webui

## Project Overview Retrieval-based-Voice-Conversion-WebUI (RVC) is a simple and easy-to-use voice conversion framework with a web interface. The README states that the base model was trained on the open-source VCTK dataset, which contains nearly 50 hours of audio, and claims there are no copyright concerns. ## Key Features - Uses top-1 retrieval to replace input source features with training set features, helping suppress timbre leakage. - Can train relatively quickly even on low-end GPUs. - Requires only a small amount of data; recommends collecting at least 10 minutes of low-noise speech. - Supports model merging via ckpt-merge to modify timbre. - Provides a simple and user-friendly web interface. - Can call pymss/MSST models to separate vocals from accompaniment. - Employs the InterSpeech2023-RMVPE pitch extraction algorithm for vocals. The README claims it mitigates the mute issue, runs fast, and uses fewer resources. - AMD/Intel GPUs use a CPU-dependent solution; Windows can use DirectML, while Linux uses CPU. ## Interface and Latency The README shows the training/inference interface and the real-time voice conversion interface. The real-time section claims an end-to-end latency of 170ms; if ASIO input/output devices are used, end-to-end latency can be reduced to 90ms, though this heavily depends on hardware driver support. ## Environment Setup This branch is targeted for Python 3.12 x64. You must first enter the repository root directory. Ubuntu 24.04 x86_64 is recommended. On Ubuntu 24.04, install python3.12, python3.12-venv, python3.12-dev, ffmpeg, unzip, libsndfile1, and libportaudio2. Then create a virtual environment and upgrade pip, setuptools, and wheel. On Windows, install Python 3.12 x64 and create a virtual environment similarly. Dependencies are selected based on hardware: - CPU, AMD, Intel: Use `requirments_cpu_py312.txt`; Windows can use DirectML, Linux uses CPU. - NVIDIA RTX 50 series: First install Torch for CUDA 12.8, then install `requirments_cu128_py312.txt`. - NVIDIA RTX 50 series and earlier: First install Torch for CUDA 11.8, then install `requirments_cu118_py312.txt`. The README provides commands to check Torch and CUDA status. The three dependency files include download sources at the top; users in mainland China can keep the default mirrors or replace them with official sources. ## Models and Runtime Directories The WebUI automatically creates runtime directories. Models must be downloaded from the Hugging Face model repository and kept in specified paths, including `assets/hubert_base`, `assets/rmvpe`, `assets/pretrained`, `assets/pretrained_v2`, `assets/pymss_weights`, `assets/weights`, `assets/indices`, and `logs/mute`. The README gives commands using `huggingface_hub` to download each component model, and notes that only Windows AMD/Intel DirectML environments also require `rmvpe.onnx`. FFmpeg is installed with system dependencies on Ubuntu; Windows users can place `ffmpeg.exe` and `ffprobe.exe` in the project root directory. ## Getting Started To start the WebUI, use `python webui.py`; on headless Ubuntu servers, use `python webui.py --noautoopen`. The default service listening port is 7865. Place your own `.pth` models in `assets/weights/` and `.index` files in `assets/indices/`. ## Reference Projects The README lists reference projects including ContentVec, VITS, HIFIGAN, Gradio, FFmpeg, Ultimate Vocal Remover, pymss, audio-slicer, and RMVPE. It states that the RMVPE pretrained model was trained and tested by yxlllc and RVC-Boss.