Search at the Cost of Sampling: Nearly-Instant Latent Space Bayesian Optimization
cs.LG
Submitted: 2026-09-16
Updated: 2026-09-16
Code: https://github.com/DonneyF/nearly-instant-lsbo
License: http://creativecommons.org/licenses/by/4.0/
The gist: Generative models are increasingly central to many de novo discovery pipelines, in which designs are generated at scale and filtered through virtual screens to determine a set of candidates to
Terminology
Abstract
Generative models are increasingly central to many de novo discovery pipelines, in which designs are generated at scale and filtered through virtual screens to determine a set of candidates to experimentally validate. While Bayesian optimization (BO) is a natural fit for this setting, as it uses past evaluations to guide future proposals, the computational overhead required for its sequential decision-making becomes a bottleneck when virtual screens are relatively cheap. We make BO practical in this regime by exploiting the unique combination of a linear model constrained to a spherical domain where high-dimensional latents concentrate. We build off recent work justifying the use of linear surrogates, while deriving nearly closed-form solutions to the surrogate modelling and acquisition problems that exploit spherical symmetry. The result is at least a 100x speedup over state-of-the art baselines, with matching or improved performance across molecular and image generation benchmarks. Altogether, our method makes BO a practical drop-in for de novo pipelines where it was previously too slow to consider.
Sources
- High-Dimensional Bayesian Optimisation with Variational Autoencoders and Deep Metric Learning
- Hyperspherical Latents Improve Continuous-Token Autoregressive Generation
- Thin-shell bounds via parallel coupling
- Detecting Out-of-Distribution Inputs to Deep Generative Models Using Typicality
- Sampling Generative Networks
- Sample-Efficient Optimisation over the Outputs of Generative Models
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks