lingbot-map

LingBot-Map is a feed-forward 3D foundation model designed for streaming 3D reconstruction from video or image sequences. It introduces a Geometric Context Transformer that unifies coordinate grounding, dense geometric cues, and long-range drift correction within a single streaming framework using anchor context, pose-reference windows, and trajectory memory. The architecture enables high-efficiency inference at approximately 20 FPS on 518x378 resolution over sequences exceeding 10,000 frames, leveraging paged KV cache attention (FlashInfer recommended, with SDPA fallback). The repository provides two main workflows: an interactive demo (`demo.py`) using a browser-based viser viewer for real-time visualization, and an offline rendering pipeline (`demo_render/batch_demo.py`) for producing headless point-cloud flythrough MP4s from long videos. Both support sky masking via ONNX segmentation models, keyframe interval strategies to manage KV cache memory, and windowed inference for sequences beyond the ~320-frame training range. Pre-trained checkpoints are available on HuggingFace and ModelScope, with evaluation benchmarks for datasets like KITTI and Oxford Spires included. Installation requires Python 3.10, PyTorch 2.8.0 with CUDA 12.8, and optional dependencies for rendering (Open3D, Kaolin, ffmpeg) and visualization. The project is licensed under Apache-2.0 and includes detailed documentation, worked examples (indoor walkthrough, outdoor drive, generated scenes), and performance tuning options for limited GPU memory.