TSGate: Timestep-Aware Gated Attention for Diffusion Transformers

arXiv:2609.34539 · cs.CV · Submitted 2026-09-28 · Read on arXiv

cs.CV

Submitted: 2026-09-28

Updated: 2026-09-28

Code: https://github.com/black-forest-labs/flux

Terminology

• Transformer — 35× in this paper · explained in 2 episodes
• Benchmark — 12× in this paper · explained in 2 episodes
• Diffusion Model — 7× in this paper · explained in 8 episodes
• Alignment — 6× in this paper · explained in 1 episode
• Diffusion Models — 6× in this paper · explained in 6 episodes
• Robustness — 6× in this paper · explained in 5 episodes
• Fidelity — 4× in this paper · explained in 2 episodes
• Aggregation — 3× in this paper · explained in 1 episode
• Attention Mechanism — 3× in this paper · explained in 2 episodes
• Joint Attention — 3× in this paper · explained in 2 episodes
• Generalization — 2× in this paper · explained in 13 episodes
• Generative Modeling — 2× in this paper · explained in 3 episodes
• Sparsity — 2× in this paper · explained in 4 episodes
• Activation Function — 1× in this paper · explained in 1 episode
• AdamW Optimizer — 1× in this paper · explained in 1 episode
• Distribution Shift — 1× in this paper · explained in 5 episodes
• Feature Selection — 1× in this paper · explained in 4 episodes
• Flow Matching — 1× in this paper · explained in 8 episodes
• Latent Diffusion Model — 1× in this paper · explained in 2 episodes
• Latent Diffusion Models — 1× in this paper · explained in 2 episodes
• Prompt Adherence — 1× in this paper · explained in 2 episodes
• Prompt Engineering — 1× in this paper · explained in 4 episodes
• Rectified Flow — 1× in this paper · explained in 3 episodes
• Regularization — 1× in this paper · explained in 3 episodes
• Reinforcement Learning — 1× in this paper · explained in 4 episodes
• Semantic Alignment — 1× in this paper · explained in 3 episodes
• Transformer Architecture — 1× in this paper · explained in 2 episodes

Sources

• Qwen2.5-VL Technical Report
• Scaling Diffusion Transformers to 16 Billion Parameters
• Attention Sink Forges Native MoE in Attention Layers: Sink-Aware Training to Address Head Collapse
• Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformers
• Classifier-Free Diffusion Guidance
• ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment
• Text Template Tokens Are Implicit Semantic Registers in Diffusion Transformers
• Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding
• Decoupled Weight Decay Regularization
• Attention Sink in Transformers: A Survey on Utilization, Interpretation, and Mitigation
• Retentive Network: A Successor to Transformer for Large Language Models
• Seedance 2.0: Advancing Video Generation for World Complexity
• Wan: Open and Advanced Large-Scale Video Generative Models
• Attention Sinks in Diffusion Transformers: A Causal Analysis
• Root Mean Square Layer Normalization

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models