Diffusion Models Adapt to Low-Dimensional Structure Under Flexible Coefficient Choices
Changxiao Cai, Yuchen Jiao, Gen Li
stat.ML, cs.LG, math.ST
Submitted: 2026-06-22
License: http://creativecommons.org/licenses/by/4.0/
The gist: Diffusion models are known to exploit unknown low-dimensional structure to accelerate sampling.
Terminology
Abstract
Diffusion models are known to exploit unknown low-dimensional structure to accelerate sampling. However, existing convergence theory under low-dimensional data structure has largely focused on update rules with narrowly prescribed coefficient choices. This raises a fundamental question: is adaptation to low-dimensional structure sensitive to the precise choice of update coefficients? In this paper, we show that such adaptation is a robust property of diffusion models. For a broad class of update coefficients, we prove that (k/epsilon) iterations suffice to generate an epsilon-accurate sample in total variation (TV) distance, independently of the ambient dimension. Our framework substantially broadens the class of diffusion samplers known to enjoy low dimensional adaptation and applies to several commonly used methods in practice. These results provide a theoretical justification for the empirical effectiveness of diffusion samplers across different coefficient choices when applied to structured, high-dimensional data.
Sources
- Convergence of Diffusion Models Under the Manifold Hypothesis in High-Dimensions
- Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models
- Nearly $d$-Linear Convergence Bounds for Diffusion Models via Stochastic Localization
- Minimax Optimality of the Probability Flow ODE for Diffusion Models
- Confidence-Based Decoding is Provably Efficient for Diffusion Language Models
- Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions
- Optimal Inference Schedules for Masked Diffusion Models
- Efficient Sampling with Discrete Diffusion Models: Sharp and Adaptive Guarantees
- Optimal estimation of a factorizable density using diffusion models with ReLU neural networks
- Faster Diffusion Sampling with Randomized Midpoints: Sequential and Parallel
- Convergence Analysis of Probability Flow ODE for Score-based Generative Models
- Reverse Transition Kernel: A Flexible Framework to Accelerate Diffusion Inference
- Denoising diffusion probabilistic models are optimally adaptive to unknown low dimensionality
- Instance-dependent Convergence Theory for Diffusion Models
- Provable Acceleration for Diffusion Models under Minimal Assumptions
- Dimension-Free Convergence of Diffusion Models for Approximate Gaussian Mixtures
- Accelerating Convergence of Score-Based Diffusion Models, Provably
- Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative Models
- Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Models
- O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptions
Related papers
- Behavior of prediction performance metrics with rare events
- Optimal Estimation of Generic Dynamics by Path-Dependent Neural Jump ODEs
- A Posterior-Dynamics Framework for Imaging Inverse Problems with Pretrained Diffusion Priors
- One Permutation Is All You Need: Fast, Deterministic Feature Importance and Model Stress-Testing
- Online Conformal Prediction for Non-Exchangeable Panel Data
- Deep Time-Series Forecasting in 10 Years: A Survey