PARK: Accurate Block Retrieval for Sparse Attention in Video Diffusion Transformers

arXiv:2609.38978 · cs.CV · Submitted 2026-09-30 · Read on arXiv

cs.CV

Submitted: 2026-09-30

Updated: 2026-09-30

Code: https://github.com/mit-han-lab/Block-Sparse-Attention

Terminology

• Clustering — 67× in this paper · explained in 2 episodes
• Sparse Attention — 67× in this paper · explained in 5 episodes
• Transformer — 40× in this paper · explained in 2 episodes
• Aggregation — 13× in this paper · explained in 1 episode
• Sparsity — 9× in this paper · explained in 4 episodes
• Query-Response Key Clustering — 7× in this paper
• Diffusion Model — 6× in this paper · explained in 8 episodes
• Diffusion Models — 6× in this paper · explained in 6 episodes
• Per-query Attention-Mass Aggregation — 5× in this paper
• Accuracy — 3× in this paper · explained in 1 episode
• Benchmark — 3× in this paper · explained in 2 episodes
• Trace — 3× in this paper · explained in 2 episodes
• Ablation — 2× in this paper · explained in 1 episode
• Ablation study — 2× in this paper · explained in 1 episode
• Consistency — 2× in this paper · explained in 3 episodes
• Fidelity — 2× in this paper · explained in 2 episodes
• Adaptation — 1× in this paper · explained in 1 episode
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Latent Space — 1× in this paper · explained in 7 episodes

Sources

• DFSAttn: Dynamic Fine-grained Sparse Attention for Efficient Video Generation
• HunyuanVideo: A Systematic Framework For Large Video Generative Models
• Compact Attention: Exploiting Structured Spatio-Temporal Sparsity for Fast Video Generation
• Radial Attention: $O(n\log n)$ Sparse Attention with Energy Decay for Long Video Generation
• Attention Sparsity is Input-Stable: Training-Free Sparse Attention for Video Generation via Offline Sparsity Profiling and Online QK Co-Clustering
• AdaCluster: Adaptive Query-Key Clustering for Sparse Attention in Video Generation
• Wan: Open and Advanced Large-Scale Video Generative Models
• Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity
• Bidirectional Sparse Attention for Faster Video Diffusion Training
• TurboDiffusion: Accelerating Video Diffusion Models by 100-200 Times

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models