liveportrait

LivePortrait is the official PyTorch implementation of the paper "LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control" (arXiv 2407.03168), developed by researchers at Kuaishou Technology, USTC and Fudan University. The project animates still portraits or portrait videos by transferring motion from a driving video or motion template. Key capabilities described in the README: - Portrait animation for humans, plus an Animals mode for cats and dogs (Animals mode requires X-Pose and is tested only on Linux/Windows with NVIDIA GPU). - Image driven mode and video driven mode (portrait video editing, v2v). - Stitching and retargeting control, regional control, and precise portrait editing in the Gradio interface. - Motion template files (.pkl) that can speed up inference and help protect privacy by avoiding distribution of driving videos. - Driving video auto-cropping with adjustable scale and offset options. - Audio and video concatenation support. - A Gradio web interface for humans and a separate one for animals, with options such as server port, sharing, and torch.compile acceleration (not supported on Windows/macOS). - A speed evaluation script for measuring per-module inference performance. Installation and usage: - Requires git, conda and FFmpeg; a conda environment with Python 3.10 is created, then dependencies are installed from requirements.txt (or requirements_macOS.txt for Apple Silicon). - Pretrained weights are downloaded from HuggingFace (with an hf-mirror alternative) or from Google Drive/Baidu Yun, and placed in ./pretrained_weights. - Inference is run via inference.py for humans and inference_animals.py for animals; source and driving inputs are specified with -s and -d. - macOS with Apple Silicon is supported for Humans mode, noted as potentially much slower than an RTX 4090. The README also lists numerous community-developed projects and integrations, including ComfyUI nodes, Stable Diffusion WebUI extensions, FaceFusion, TensorRT-based faster versions, and various HuggingFace Spaces and playgrounds. It acknowledges prior work such as FOMM, Open Facevid2vid, SPADE, InsightFace and X-Pose, and includes an ethics note about deepfake risks and responsible use.