open-sora

Open-Sora is an open-source project dedicated to democratizing efficient video production by providing accessible models, tools, and implementation details. The project offers a full pipeline for video generation, including data preprocessing, training acceleration (utilizing ColossalAI), and inference. The latest release, Open-Sora 2.0, features an 11B parameter model. According to the project's technical report, this model was trained with a budget of approximately $200K and achieves on-par performance with other open-source models like HunyuanVideo (11B) and Step-Video (30B) on the VBench benchmark and in human preference evaluations. The project also maintains previous versions (1.0 through 1.3) on separate branches, documenting an iterative progression in capabilities such as 3D-VAE, rectified flow, and various spatial-temporal architectures. Key capabilities and features include: - Text-to-image, text-to-video, image-to-video, and video-to-video generation. - Support for variable video lengths (from 2 seconds up to 16 seconds or infinite time) and resolutions ranging from 144p to 720p, with flexible aspect ratios. - Fully open-source checkpoints and training scripts, allowing users to train or fine-tune their own models. - Detailed technical reports documenting the architecture, training processes, and evaluation metrics for each version. - A Gradio-based demo hosted on Hugging Face Spaces for interactive testing. The project provides comprehensive documentation for training, evaluating video autoencoders, and deploying the models, aiming to simplify the complexities of video generation for content creators and researchers.