wan2.2

Wan2.2 is an open-source video generative model suite developed by Wan-AI. It introduces a Mixture-of-Experts (MoE) architecture into video diffusion models, separating the denoising process across timesteps with specialized expert models to increase capacity while maintaining computational cost. The suite supports multiple generation tasks: - Text-to-Video (T2V-A14B): Generates video from text prompts at 480P and 720P resolutions. - Image-to-Video (I2V-A14B): Generates video from a reference image and text prompt. - Text-Image-to-Video (TI2V-5B): A 5B parameter model using a Wan2.2-VAE with 16x16x4 compression ratio, supporting 720P at 24fps. This model can run on consumer-grade GPUs with at least 24GB VRAM (e.g., RTX 4090). - Speech-to-Video (S2V-14B): An audio-driven cinematic video generation model that accepts audio input, a reference image, and optional text prompt. Supports pose-driven generation via a pose video parameter. Can integrate CosyVoice for text-to-speech synthesis. - Character Animation and Replacement (Animate-14B): Takes a video and character image as input, generating videos in either animation mode (character mimics input video motion) or replacement mode (replaces character in input video). The repository provides single-GPU and multi-GPU inference scripts using PyTorch FSDP and DeepSpeed Ulysses for acceleration. Prompt extension is supported via Dashscope API or local Qwen models to enrich generated video details. Model weights are available on Hugging Face and ModelScope. Wan2.2 has been integrated into ComfyUI and Hugging Face Diffusers. The community has built related projects including LightX2V (lightweight inference framework), FastVideo (distilled models with sparse attention), Cache-dit (cache acceleration), DiffSynth-Studio (low-VRAM offload, FP8 quantization, LoRA training), and Kijai's ComfyUI WanVideoWrapper.