MGD: Moment Guided Diffusion for Maximum Entropy Generation
stat.ML, cs.LG
Submitted: 2026-02-19
Updated: 2026-09-06
License: http://creativecommons.org/licenses/by/4.0/
The gist: Generating samples from limited information is a fundamental problem across scientific domains.
Terminology
Abstract
Generating samples from limited information is a fundamental problem across scientific domains. Classical maximum entropy methods provide principled uncertainty quantification from moment constraints but require sampling via MCMC or Langevin dynamics, which typically exhibit exponential slowdown in high dimensions. In contrast, generative models based on diffusion and flow matching efficiently transport noise to data but offer limited theoretical guarantees and can overfit when data is scarce. We introduce Moment Guided Diffusion (MGD), which combines elements of both approaches. Building on the stochastic interpolant framework, MGD samples maximum entropy distributions by solving a stochastic differential equation that guides moments toward prescribed values in finite time, thereby avoiding slow mixing in equilibrium-based methods. We formally obtain, in the large-volatility limit, convergence of MGD to the maximum entropy distribution and derive a tractable estimator of the resulting entropy computed directly from the dynamics. Applications to financial time series, turbulent flows, and cosmological fields using wavelet scattering moments yield estimates of negentropy for high-dimensional multiscale processes.
Sources
- Flow Matching for Generative Modeling
- Building Normalizing Flows with Stochastic Interpolants
- Stochastic Interpolants: A Unifying Framework for Flows and Diffusions
- The Principles of Diffusion Models
- Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow
- Statistical Efficiency of Score Matching: The View from Isoperimetry
- Bigger Isn't Always Memorizing: Early Stopping Overparameterized Diffusion Models
- Propagation of chaos: a review of models, methods and applications. I. Models and methods
Related papers
- Behavior of prediction performance metrics with rare events
- Optimal Estimation of Generic Dynamics by Path-Dependent Neural Jump ODEs
- A Posterior-Dynamics Framework for Imaging Inverse Problems with Pretrained Diffusion Priors
- One Permutation Is All You Need: Fast, Deterministic Feature Importance and Model Stress-Testing
- Online Conformal Prediction for Non-Exchangeable Panel Data
- Deep Time-Series Forecasting in 10 Years: A Survey