lingbot-world

LingBot-World is an open-source world simulator and video generation framework built upon the Wan2.2 architecture. It is designed to advance open-source world models by offering high-fidelity simulations, robust dynamics across diverse visual styles (such as realism, scientific contexts, and cartoons), and long-term memory consistency over minute-level horizons. The model supports real-time interactivity, achieving low latency (under 1 second) at 16 frames per second. Key Features: - **High-Fidelity & Diverse Environments**: Capable of simulating complex environments across various visual domains. - **Long-Term Memory & Consistency**: Maintains contextual consistency over long video horizons. - **Real-Time Interactivity**: Supports interactive controls with low latency, suitable for gaming, content creation, and robot learning. - **Flexible Control Signals**: Can be guided by camera poses, action sequences, or user-friendly action strings (e.g., movement and camera direction commands), or run unconditionally. Models and Tools: - **LingBot-World-Base (Cam)**: Camera pose-controlled model available in 480P and 720P resolutions. - **LingBot-World-Base (Act)**: Action-controlled model supporting text-based action strings. - **LingBot-World-Fast**: An accelerated version utilizing causal inference and KV caching for efficient, chunk-by-chunk frame generation. - **Quantized Model**: A 4-bit quantized version is available for users with limited GPU memory, though it is for inference only and may exhibit minor visual degradation. Technical Requirements & Usage: The project requires PyTorch (>= 2.4.0) and flash attention. Inference scripts support multi-GPU execution via `torchrun`. Model weights can be downloaded from HuggingFace or ModelScope. The repository provides example scripts for image-to-video generation with various configuration options. Note: This repository is no longer actively maintained. The authors have transitioned development to the newer repository, LingBot-World-Infinity (v2).