colossalai

Colossal-AI is an open-source project from HPC-AI Tech that provides a collection of parallel components for distributed deep learning, aiming to make large AI models cheaper, faster, and more accessible. It lets users write distributed training and inference code with minimal changes, largely driven by configuration files. Key capabilities described in the README include parallelism strategies such as data parallelism, pipeline parallelism, 1D/2D/2.5D/3D tensor parallelism, sequence parallelism, ZeRO, and auto-parallelism. It also includes heterogeneous memory management via PatrickStar. The project highlights real-world applications and examples: Open-Sora for video generation models, Colossal-LLaMA-2 for domain-specific LLM training, ColossalChat with a complete RLHF pipeline, acceleration of Stable Diffusion training and inference, and FastFold for AlphaFold protein structure work. Parallel training demos cover LLaMA 1/2/3, MoE, GPT-3, GPT-2, BERT, PaLM, OPT, ViT, and recommendation models, plus single-GPU demos and inference components such as Colossal-Inference, Grok-1 support, and SwiftInfer. Installation is available via PyPI or from source, and Docker usage is documented. The README also links to documentation, examples, a forum, and community channels. Note that the README contains promotional content for HPC-AI Cloud GPU and model API services; those are commercial offerings rather than part of the open-source library itself.