SparSP: Exploiting Communication Sparsity for Sequence-Parallel Video DiTs

arXiv:2609.32197 · cs.DC, cs.LG, cs.PF · Submitted 2026-09-26 · Read on arXiv

cs.DC, cs.LG, cs.PF

Submitted: 2026-09-26

Updated: 2026-09-26

Code: https://github.com/MiniMax-H3/MiniMax-H3

Terminology

• Sparse Attention — 34× in this paper · explained in 5 episodes
• Sparsity — 31× in this paper · explained in 4 episodes
• Diffusion Model — 18× in this paper · explained in 8 episodes
• Diffusion Models — 16× in this paper · explained in 6 episodes
• Transformer — 15× in this paper · explained in 2 episodes
• Communication Overhead — 3× in this paper · explained in 2 episodes
• Core — 3× in this paper · explained in 2 episodes
• Generative Model — 3× in this paper · explained in 2 episodes
• Generative Models — 3× in this paper · explained in 2 episodes
• Aggregation — 2× in this paper · explained in 1 episode
• Conference on Computer Vision and Pattern Recognition — 2× in this paper
• Decoupling — 2× in this paper · explained in 2 episodes
• Generative AI — 2× in this paper · explained in 2 episodes
• Redundancy — 2× in this paper · explained in 2 episodes
• Ablation — 1× in this paper · explained in 1 episode
• Ablation study — 1× in this paper · explained in 1 episode
• Adam — 1× in this paper · explained in 2 episodes
• Benchmark — 1× in this paper · explained in 2 episodes
• Consistency — 1× in this paper · explained in 3 episodes
• Distillation — 1× in this paper · explained in 2 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Foundation Model — 1× in this paper · explained in 3 episodes
• KV cache — 1× in this paper · explained in 3 episodes
• NVIDIA Collective Communications Library — 1× in this paper
• Quantization — 1× in this paper · explained in 12 episodes
• Robustness — 1× in this paper · explained in 5 episodes
• Transformer Models — 1× in this paper · explained in 2 episodes

Sources

• SpComm3D: A Framework for Enabling Sparse Communication in 3D Sparse Kernels
• LTX-2: Efficient Joint Audio-Visual Foundation Model
• LTX-Video: Realtime Video Latent Diffusion
• HunyuanVideo: A Systematic Framework For Large Video Generative Models
• FVAttn: Adaptive Sparse Attention with Runtime Load Balancing for Video Generation
• Seedance 2.0: Advancing Video Generation for World Complexity
• FlexCache: Flexible Approximate Cache System for Video Diffusion
• Wan: Open and Advanced Large-Scale Video Generative Models
• PecSched: Preemptive and Efficient Cluster Scheduling for LLM Inference
• SGLang: Efficient Execution of Structured Language Model Programs

Related papers

• iScheduler: Reinforcement Learning-Driven Continual Optimization for Large-Scale Resource Investment Problems
• SAMM: Sharded Automated Market Maker
• InferScale: GPU-Native KV Injection for Personalized LLM Serving
• Vigil: Accountable Liveness against Selective Silence
• Steelhead: Interleaving Partially Synchronous and Asynchronous Commit Rules on a Shared DAG
• Pushing CPU Speech Synthesis to the Wall: Extreme Inference Tuning under Serverless Architecture and Billing