Denoising growth complexity: Data geometry and certified schedules for diffusion sampling
Martin J. Wainwright
math.ST, cs.LG, math.NA, stat.ML
Submitted: 2026-07-28
License: http://creativecommons.org/licenses/by/4.0/
The gist: Two central challenges in diffusion-based sampling are the theoretical one of understanding their remarkable effectiveness even in high-dimensional settings, and the practical one of designing
Terminology
Abstract
Two central challenges in diffusion-based sampling are the theoretical one of understanding their remarkable effectiveness even in high-dimensional settings, and the practical one of designing algorithms with certified performance guarantees. We show that these questions are intimately connected via the denoising growth complexity. It is a geometric measure defined by a log-time weighted integral of the derivative of the denoising mean-squared error along the Gaussian heat flow. We show how the increments lead to a simple and explicit bound on the KL error of an Euler scheme applied to the stochastic innovations representation. The bound is local along the path: each step is controlled by the corresponding increment and its relative stepsize. This structure allows us to derive KL sampling guarantees for optimized stepsize schedules, both in a simpler single-block setting and in a more refined K-block setting. The function has a natural martingale structure, which we exploit to develop fully data-certified versions of these algorithms. It also admits information-theoretic upper bounds in terms of covariance, rate distortion, metric entropy, and the Poincar'e constant, thereby recovering and sharpening a range of existing diffusion-sampling guarantees, as well as giving new results. In log heat-time, the fine partition limit is governed by an integral involving the square root of the density, whereas a single-block schedule depends on its ordinary integral. This comparison precisely characterizes when adaptation to data geometry yields substantial computational gains, including logarithmic-to-constant separations for simple Gaussian mixture models.
Sources
- Convergence of Diffusion Models Under the Manifold Hypothesis in High-Dimensions
- Error Bounds for Flow Matching Methods
- High-accuracy sampling for diffusion models and log-concave distributions
- The probability flow ODE is provably fast
- KL Convergence Guarantees for Score diffusion models under minimal data assumptions
- Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data
- Minimax Optimality of the Probability Flow ODE for Diffusion Models
- An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimization
- High-accuracy and dimension-free sampling with diffusions
- Learning Mixtures of Gaussians Using Diffusion Models
- Provable Acceleration for Diffusion Models under Minimal Assumptions
- Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative Models
- A Simple Proof of the Mixing of Metropolis-Adjusted Langevin Algorithm under Smoothness and Isoperimetry
- Faster Diffusion Models via Higher-Order Approximation
- Sampling, Diffusions, and Stochastic Localization
- Diffusion Models: A Comprehensive Survey of Methods and Applications
Related papers
- Conformal Prediction for Dyadic Regression Under Complex Missingness
- Bentkus-type asymptotic e-values
- High-Dimensional Asymptotics of Differentially Private PCA
- KL Convergence Guarantees for Score diffusion models under minimal data assumptions
- Geometric bias in eigenspace perturbation under random heterogeneous noise
- On the Asymptotic Inadmissibility of Double Machine Learning Estimators Under Structure-Agnostic Models