Quenched Ensemble Sampling
stat.ML, cs.LG, stat.CO
Submitted: 2026-09-14
Updated: 2026-09-14
Comments: 27 pages, 8 figures
Code: https://github.com/yallup/quenched
License: http://creativecommons.org/licenses/by/4.0/
The gist: Some of the sharpest challenges in sampling from the energy functions of physical systems arise at phase transitions, where the density of states changes abruptly and many sampling algorithms stall.
Terminology
Abstract
Some of the sharpest challenges in sampling from the energy functions of physical systems arise at phase transitions, where the density of states changes abruptly and many sampling algorithms stall. Nested sampling is a particle method that traverses the density of states under a hard energy constraint and is known to be robust to such transitions, but its application in high dimension is limited by the difficulty of sampling under that constraint. In this work we introduce Quenched Ensemble Sampling, which generalises the hard constraint to a family of repulsive potentials at the energy boundary. This preserves the quenched path of monotonically decreasing energy while making the constrained target amenable to scalable gradient-based kernels. We demonstrate on synthetic models of phase transitions that our method estimates the marginal likelihood and draws posterior samples across a first-order transition where popular alternatives such as tempering fail. We apply the procedure to marginal likelihood estimation in Bayesian neural networks, enabling model comparison between network architectures. Finally, in a high-dimensional continuous lattice field theory, we show that this method traverses a first-order transition and estimates the partition function.
Sources
- Scalable Bayesian Learning with posteriors
- Adam: A Method for Stochastic Optimization
- Marginal likelihood computation for model selection and hypothesis testing: an extensive review
- On adaptive resampling strategies for sequential Monte Carlo methods
- MCMC using Hamiltonian dynamics
- Vertical-likelihood Monte Carlo
- Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets
- Non-Reversible Parallel Tempering: a Scalable Highly Parallel MCMC Scheme
- Nested Sampling with Slice-within-Gibbs: Efficient Evidence Calculation for Hierarchical Bayesian Models
- Nested Slice Sampling: Vectorized Nested Sampling for GPU-Accelerated Inference
Related papers
- Behavior of prediction performance metrics with rare events
- Optimal Estimation of Generic Dynamics by Path-Dependent Neural Jump ODEs
- A Posterior-Dynamics Framework for Imaging Inverse Problems with Pretrained Diffusion Priors
- One Permutation Is All You Need: Fast, Deterministic Feature Importance and Model Stress-Testing
- Online Conformal Prediction for Non-Exchangeable Panel Data
- Deep Time-Series Forecasting in 10 Years: A Survey