Second Order Drifting Models
Drake Brown, Yuhao Huang, Shih-Hsin Wang, Bao Wang
cs.LG, cs.AI, cs.NA, math.NA
Submitted: 2026-08-08
Updated: 2026-08-11
Comments: 20 pages, 4 figures
License: http://creativecommons.org/publicdomain/zero/1.0/
The gist: Drifting models are a recent class of one-step generative models that evolve the model distribution during training using a predefined sample-based drift field.
Terminology
Abstract
Drifting models are a recent class of one-step generative models that evolve the model distribution during training using a predefined sample-based drift field. Although they avoid iterative inference, their kernel-based drift fields induce frequency-dependent training dynamics: In the linearized regime, each Fourier mode of the density residual decays at a rate determined by the kernel spectrum, leading to slow recovery of fine-scale structure. We propose Second-Order Drifting Models, which lift drifting dynamics into phase space by augmenting generated samples with artificial velocity variables. We show that the resulting density perturbations obey accelerated second-order dynamics in Fourier space, connecting drifting models to the celebrated Nesterov acceleration from optimization theory. This provides a principled mechanism for mitigating the spectral stiffness of first-order drifting while preserving one-step inference. We derive a practical semi-implicit training algorithm and evaluate it on synthetic distribution matching, sequential data generation, and robotic control. Across these settings, the second-order drifting model improves convergence behavior and achieves competitive or superior performance over first-order drifting baselines.
Sources
- Stochastic Interpolants: A Unifying Framework for Flows and Diffusions
- Flow map matching with stochastic interpolants: A mathematical framework for consistency models
- How to build a consistency model: Learning flow maps via self-distillation
- Gradient Flow Drifting: Generative Modeling via Wasserstein Gradient Flows of KDE-Approximated Divergences
- Generative Modeling via Drifting
- Mean Flows for One-step Generative Modeling
- Variational Rectified Flow Matching
- Sinkhorn-Drifting Generative Models
- Improving Flow Matching by Aligning Flow Divergence
- RMFlow: Refined Mean Flow by a Noise-Injection Step for Multimodal Generation
- Plug-and-Play Image Restoration with Flow Matching: A Continuous Viewpoint
- Attraction, Repulsion, and Friction: Introducing DMF, a Friction-Augmented Drifting Model
- A Unified View of Score-Based and Drifting Models
- A Long-Short Flow-Map Perspective for Drifting Models
- Notes on Kullback-Leibler Divergence and Likelihood
- Consistency Models
- Flow Matching for Efficient and Scalable Data Assimilation
- Generative Drifting is Secretly Score Matching: a Spectral and Variational Perspective
- Analytical Correction for Subsampling Bias in Drifting Models
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks