Partition the Support, Reconstruct the Residual: Training-Free Sparse Attention for Video Generation and World Models

arXiv:2608.18484 · cs.CV, cs.AI, cs.LG · Submitted 2026-08-19 · Read on arXiv

cs.CV, cs.AI, cs.LG

Submitted: 2026-08-19

Updated: 2026-09-27

Terminology

• Sparse Attention — 31× in this paper · explained in 5 episodes
• Clustering — 13× in this paper · explained in 2 episodes
• World Model — 11× in this paper · explained in 7 episodes
• Sparsity — 10× in this paper · explained in 4 episodes
• Transformer — 10× in this paper · explained in 2 episodes
• Benchmark — 6× in this paper · explained in 2 episodes
• Fidelity — 5× in this paper · explained in 2 episodes
• Ablation — 4× in this paper · explained in 1 episode
• Generative Model — 4× in this paper · explained in 2 episodes
• Generative Models — 4× in this paper · explained in 2 episodes
• Calibration — 2× in this paper · explained in 7 episodes
• Diffusion Model — 2× in this paper · explained in 8 episodes
• Diffusion Models — 2× in this paper · explained in 6 episodes
• Reconstruction Error — 2× in this paper · explained in 2 episodes
• Regularization — 2× in this paper · explained in 3 episodes
• Accuracy — 1× in this paper · explained in 1 episode
• Alignment — 1× in this paper · explained in 1 episode
• Consistency — 1× in this paper · explained in 3 episodes
• Distillation — 1× in this paper · explained in 2 episodes
• Foundation Model — 1× in this paper · explained in 3 episodes
• Foundation Models — 1× in this paper · explained in 7 episodes

Sources

• Rethinking Attention with Performers
• LVSA: Training-Free Sparse Attention for Long Video Diffusion
• Veda: Scalable Video Diffusion via Distilled Sparse Attention
• DFSAttn: Dynamic Fine-grained Sparse Attention for Efficient Video Generation
• Reformer: The Efficient Transformer
• HunyuanVideo: A Systematic Framework For Large Video Generative Models
• Sol-Attn: Accelerating Video Generation Inference via On-the-Fly Attention Sparsification
• PISA: Piecewise Sparse Attention Is Wiser for Efficient Diffusion Transformers
• PSA: Pyramid Sparse Attention for Efficient Video Understanding and Generation
• Rectified SpaAttn: Revisiting Attention Sparsity for Efficient Video Generation
• Attention Sparsity is Input-Stable: Training-Free Sparse Attention for Video Generation via Offline Sparsity Profiling and Online QK Co-Clustering
• World Simulation with Video Foundation Models for Physical AI
• Cosmos 3: Omnimodal World Models for Physical AI
• Fast Autoregressive Video Diffusion and World Models with Temporal Cache Compression and Sparse Attention
• DraftAttention: Fast Video Diffusion via Low-Resolution Attention Guidance
• Wan: Open and Advanced Large-Scale Video Generative Models
• VMoBA: Mixture-of-Block Attention for Video Diffusion Models
• Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity
• XAttention: Block Sparse Attention with Antidiagonal Scoring
• CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models