Convergence rates for generative drifting flows: fixed-scale obstructions and multihead acceleration
cs.LG
Submitted: 2026-09-14
Updated: 2026-09-14
License: http://creativecommons.org/licenses/by/4.0/
The gist: Drifting models offer a promising route to faster generative AI: they perform gradual transport during training, while generating new samples in a single step.
Terminology
Abstract
Drifting models offer a promising route to faster generative AI: they perform gradual transport during training, while generating new samples in a single step. This paper asks whether the underlying drifting process can converge rapidly to a target distribution under ideal conditions, before finite-data or optimization effects are introduced. We show that its convergence rate depends critically on how it handles spatial scale. With a single fixed resolution, fine-scale features of the target can become nearly invisible, leading to extremely slow convergence. We introduce a multihead approach that combines scale-normalized information across a continuum of resolutions. We prove that this multihead approach restores exponential convergence near standard reference distributions. These results identify fixed resolution as a key bottleneck and provide a simple route to faster one-step generative models.
Sources
- Generative Modeling via Drifting
- Gradient Flow Drifting: Generative Modeling via Wasserstein Gradient Flows of KDE-Approximated Divergences
- Generative Drifting is Secretly Score Matching: a Spectral and Variational Perspective
- On the Wasserstein Gradient Flow Interpretation of Drifting Models
- A Unified View of Score-Based and Drifting Models
- A Long-Short Flow-Map Perspective for Drifting Models
- Drifting Fields are not Conservative
- Kernel-Gradient Drifting Models
- Identifiability and Stability of Generative Drifting in the Companion-Elliptic Kernel Family
- Sinkhorn-Drifting Generative Models
- Second Order Drifting Models
- Learning Monge maps with constrained drifting models
- Regularity of the score function in generative models
- Smooth transport map via diffusion process
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks