sparkvsr

SparkVSR is an ECCV 2026 research project for interactive video super-resolution. Instead of treating restoration as a one-way black-box process, it uses sparse high-resolution keyframes as a user-controllable signal. The model propagates visual information from those keyframes through the video while remaining grounded in the original low-resolution motion. The method uses a keyframe-conditioned, latent-to-pixel two-stage training pipeline built on CogVideoX1.5-5B-I2V. Stage 1 performs latent-space keyframe-conditioned adaptation, and Stage 2 performs pixel-space detail refinement. The repository provides inference and training code, Stage-1 and Stage-2 weights on Hugging Face, dataset preparation scripts, benchmark instructions, and a ComfyUI implementation. Inference offers three reference modes. The API mode can use the fal-ai/nano-banana-pro/edit endpoint to generate restored keyframes. The PiSA-SR mode uses the separate open-source PiSA-SR project as the keyframe generator. The no-reference mode performs blind restoration without external keyframes and is presented mainly as a fallback or baseline. Users can set keyframe indices, reference guidance strength, and a 4x upscale factor. The documentation notes that reference indices must be more than four frames apart and recommends using only the first frame for very short clips. The README reports support for flexible keyframe selection, including manual selection, codec I-frame extraction, and random sampling. It also describes a guidance mechanism that balances adherence to supplied keyframes with blind restoration when references are missing or imperfect. The authors demonstrate the approach beyond standard super-resolution on tasks such as old-film restoration and video style transfer. Setup requires Python 3.10 or newer, PyTorch 2.5 or newer, Diffusers, and the packages listed in requirements.txt. Training is documented for four A100 GPUs. Training data guidance covers HQ-VSR and DIV2K-HR, while evaluation guidance covers UDM10, SPMCS, YouHQ40, RealVSR, and MovieLQ. The unified evaluation script can use external DOVER and FastVQA/FasterVQA installations, with results written to JSON. Only the Stage-2 checkpoint is intended for final inference.