Amortized quadrature for posterior expectations in inverse problems

arXiv:2606.15871 · stat.CO, cs.LG, stat.ML · Submitted 2026-06-14 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Amortized quadrature for posterior expectations in inverse problems".

Jane: Detailed Research Summary: Amortized Mean-Shift Interacting Particles for Bayesian Inference in Inverse Problems This research introduces a novel framework, Amortized Mean-Shift Interacting Particles,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: We’re looking at "Amortized quadrature for posterior expectations in inverse problems" today, and the title itself tells us immediately that they are solving a major problem with how we calculate things like posterior means when the underlying physics demands expensive simulations for every single trial.

Jane: It really does sound heavy, Tom; "amortized quadrature" suggests they're finding a way to get high-quality estimates without the usual brute-force sampling nightmare. I think for our listeners, we need to explain that this is about making those complex calculations much faster and more reliable.

Lu: What’s striking about the authors is how they aren't just tweaking existing methods; they are proposing a fundamental shift in how we estimate posterior means or risks across a stream of observations. This changes the whole approach to integration for high-dimensional problems, which is truly significant.

Meng: From an engineering standpoint, I'm curious about the practical takeaway here; does this mean we can actually run these complex physical models faster in real-time applications instead of waiting hours for a single Monte Carlo run?

Lalam: This paper looks like it has the potential to dramatically improve how we handle uncertainty quantification across almost any domain, making the results much more robust. It’s a really powerful concept for structuring complex information in ways that are much more scalable than traditional methods.

The paper's summary: Tom: The main point here is that standard Monte Carlo averaging struggles because you need too many samples when each one requires solving a complicated forward model, so they introduce designed sets of signed-weight nodes instead of random samples to estimate those integrals more accurately.

Jane: That makes sense; so instead of guessing the area under the curve by throwing darts randomly, they're strategically placing smart points that give them a better picture of where the important mass actually lies in that high-dimensional space.

Lu: The core idea is this "amortized mean-shift interacting particles" map, which they claim can emit both those quadrature nodes and some samples in just one pass from an observation without needing to constantly optimize or re-evaluate the posterior density or score at every step. This is a major shortcut because it cuts down on the repeated heavy computation.

Meng: That single forward pass idea is what interests me; if it truly cuts down on repeated computations, that could translate directly into significant speed gains for our infrastructure when dealing with massive datasets, which is exactly what we aim for in production.

Lalam: From my perspective, the fact that this map learns to integrate by only looking at samples—not needing the density or score explicitly—is really impressive. That kind of efficiency in learning is exactly what we need to see in future AI applications to make them more sustainable and scalable because it shows a pathway toward much leaner models.

The paper's improvements: Tom: Now we get into the actual improvements they claim; they show that this method isn't just a slight tweak, but it actually beats standard Monte Carlo estimators at every budget level because of how they handle those nodes and the two levers they use.

Jane: So, if we look at that comparison, it means even just tweaking where those designed points are placed—moving them strategically—can provide a significant boost in accuracy compared to just reweighting what you already have. That’s a tangible improvement for the quality of the results we get in practice.

Lu: The paper also addresses the high-dimensional wall problem; they use something called a posterior-whitened, dimension-aware kernel to make sure their method performs well even when dealing with thousands of coefficients in groundwater models, which is where traditional methods usually break down.

Meng: I need to know more about that whitening part; if they can handle high dimensions without losing information due to the curse of dimensionality, then this moves this from a theoretical curiosity to something we could actually deploy on massive physical simulations.

Lalam: It’s incredible that they manage to integrate below the Monte Carlo floor at every budget, which proves it’s not just an incremental gain; it's fundamentally better than simply drawing more independent samples. That level of theoretical guarantee is what really excites me about its potential impact on the field because it shows a new way forward for uncertainty quantification.

Conclusion: Tom: So we wrap up our discussion on "Amortized quadrature for posterior expectations in inverse problems," and the main point is that this paper replaces expensive Monte Carlo sampling with a deterministic, single-pass quadrature estimate that provides better accuracy than random drawing at every step. It’s a real game-changer for efficiency.

Jane: Exactly; it gives us a new way to get reliable posterior means and risks without needing thousands of computationally expensive samples, which is fantastic news for anyone dealing with complex physical modeling in AI inference.

Lu: I think the implication is that we're moving towards a paradigm where we can trust the integration of posteriors in inverse problems much more reliably, especially as models get more intricate.

Meng: For practical deployment, this means we could see real-time uncertainty quantification for complex physical simulations, which is a huge step toward building truly autonomous systems capable of handling messy real-world data.

Lalam: I feel really optimistic about this; the ability to build such efficient and robust tools that work across different posterior types shows how transformative this research can be for the entire AI ecosystem.

Tom: Fantastic discussion, everyone! We’ve covered a lot today on "Amortized quadrature for posterior expectations in inverse problems." Keep an eye on arXiv for more breakthroughs!

Jane: And we'll be right back after the break with something else fascinating.

Ali Siahkoohi

Department of Computer Science, University of Central Florida

stat.CO, cs.LG, stat.ML

Submitted: 2026-06-14

Updated: 2026-09-25

Code: https://github.com/alisiahkoohi/amortized-msip

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 89/100

The gist: This research introduces a novel framework, Amortized Mean-Shift Interacting Particles, designed to efficiently compute posterior expectations—such as posterior means, tail probabilities, and

Key concepts

Amortized quadrature
This is a method that estimates integrals, like posterior expectations, without needing brute-force sampling. It uses designed sets of signed-weight nodes strategically placed to get high-quality estimates faster than traditional Monte Carlo methods.
Amortized Mean-Shift Interacting Particles
This is the core idea of the research. It is a framework that can emit both quadrature nodes and samples in just one pass from an observation, avoiding the need to constantly optimize or re-evaluate the posterior density during computation.
Monte Carlo averaging
Standard Monte Carlo averaging struggles when each sample requires solving a complicated forward model, necessitating too many samples. This new method replaces random sampling with strategic node placement to estimate integrals more accurately.
Posterior-whitened, dimension-aware kernel
This component addresses the high-dimensional wall problem by using a specific kernel. It ensures the method performs well even when dealing with thousands of coefficients in complex models, preventing information loss due to high dimensions.

Terminology

Summary

This research introduces a novel framework, Amortized Mean-Shift Interacting Particles, designed to efficiently compute posterior expectations—such as posterior means, tail probabilities, and risks—across a stream of observations in inverse problems. The core motivation is to overcome the prohibitive computational cost associated with standard Monte Carlo (MC) methods when each sample requires solving a complex forward model (e.g., partial differential equations).

The standard approach for estimating integrals over a posterior distribution rho involves drawing independent samples and averaging them. This Monte Carlo average suffers from an error that decays only as the square root of the sample size (O(1/sqrt N)). For high-dimensional, physics-based posteriors (like those arising in groundwater modeling with thousands of coefficients), achieving sufficient accuracy demands an impractically large number of samples, as each sample necessitates a costly forward simulation.

The paper proposes replacing random sampling with a designed set of weighted nodes that constitute a deterministic quadrature. This quadrature estimates the required integrals more accurately than an equal number of independent samples, achieving a Pareto improvement on the Monte Carlo estimator rather than competing with it by simply drawing more samples.

The key insight is to move away from methods that require a fresh solve or score evaluation at every step (like traditional mean-shift interacting particles) toward an amortized construction.

  1. Weighted Nodes (Quadrature): Instead of independent samples, the method generates a small set of signed-weight nodes. These nodes are optimized to integrate the posterior distribution rho accurately.

  2. Amortization via Learned Maps: The crucial innovation is an amortized mean-shift interacting particles map. This is a learned function that performs two tasks in a single forward pass:

  • It emits the set of weighted quadrature nodes based on an observation.

  • It provides a handful of posterior samples for drawing from.

  1. Training Strategy: The map is trained efficiently using only joint parameter-observation samples and a reference posterior (which can be sampled from any admissible source, such as a conditional normalizing flow or an empirical Nadaraya–Watson estimator). The map learns to integrate the target posterior solely through these samples, without explicitly learning the density or score of the target itself.

  2. Generalization: Once trained, this single map generalizes across unseen observations and integrands at any requested node budget.

The paper establishes several rigorous theoretical bounds demonstrating its superiority:

  • Superior Integration Accuracy: The construction integrates more accurately than the same number of independent samples at every node budget.

  • Provable No Worse Than MC (Reweighting): By reweighting the nodes, the method achieves an error bound that is provably no worse than equal weights in a Monte Carlo setting (Proposition 2).

  • Empirical Improvement via Node Movement: Moving the nodes further improves the empirical integration error.

  • High-Dimensional Efficiency: The method effectively addresses the high-dimensional wall encountered by standard kernel methods (like isotropic kernels) where pairwise distances concentrate, causing geometric contrast to vanish. A posterior-whitened, dimension-aware kernel is used to mitigate this issue, allowing the method to perform well even in high dimensions (Section 7).

  • Efficiency: The construction is designed for a single forward pass and operates across a stream of observations.

  • Score-Free Operation: The map reads its reference posterior only through sample-estimated kernels (like the Nadaraya–Watson estimator) and self-affinity, never its density or score.

  • Quadrature Mean Coincidence: The mean of the resulting quadrature coincides with the flow posterior mean to a relative error near zero.

  • Target Fidelity: The one-pass forward pass integrates the achieved posterior faithfully, reproducing the coarse structure of the flow posterior while omitting high-frequency detail through limited-angle truncation.

The paper frames this as solving an integration problem (the deliverable of Section 1) whose cost is the variance of Monte Carlo estimators. It replaces random sampling with a designed set of weighted nodes, whose worst-case integration error is a single tractable quantity. The construction leverages the concept of mean-shift interacting particles but introduces an amortized, learned map that reads its reference only through samples to achieve this quadrature efficiently across multiple observations. Ultimately, it provides a powerful tool for high-dimensional inverse problems by offering a deterministic, single-pass integration method that outperforms independent sampling methods at every budget.

Improvements for AI systems

Here are specific improvements to AI systems that can be derived from the Amortized Mean-Shift Interacting Particles paper:

  1. Replace computationally expensive Monte Carlo (MC) integration for posterior expectations with a deterministic, single forward-pass quadrature estimate. In practice, this means calculating posterior expectations (like mean or risk) without needing thousands of independent samples.

  2. Implement an Amortized Quadrature Map that takes an observation and a set of independent samples as input to emit signed-weight nodes in one pass, rather than requiring a per-observation optimization or evaluating the posterior score repeatedly.

  3. Utilize the learned map to perform efficient uncertainty quantification (UQ) on posterior expectations by leveraging the move lever (node repositioning), which empirically lowers integration error beyond what is possible through simple reweighting.

  4. Apply this construction across diverse posterior types, including closed-form Gaussians, physics-based models (like groundwater permeability fields), learned conditional flows, and empirical measures like Nadaraya–Watson estimates.

  5. Enable the system to operate effectively in high-dimensional spaces where traditional isotropic kernel methods fail by incorporating posterior-whitened metrics (Mahalanobis kernels) and rank-truncated informed subspaces, allowing the model to focus on relevant structural information rather than ambient noise.

  6. Develop a conditional inference pipeline where an observation (e.g., a limited-angle tomography projection) allows the system to generate a posterior whose shape varies with that observation, enabling observation-varying conditional integration of complex functionals.

  7. Create an optional refinement stage for the quadrature using either more samples or an available posterior score to further reduce error, allowing practitioners to trade off computational budget for accuracy on a per-query basis.

These improvements result in AI systems that can:

  1. Perform high-stakes Bayesian inference (e.g., in seismic imaging, medical diagnostics, or complex physical modeling) with significantly reduced computational cost and variance compared to traditional Monte Carlo methods.

  2. Provide more accurate posterior mean estimates and risk assessments at every computational budget level, moving beyond the limitations of standard MCMC or basic sampling methods.

  3. Quantify uncertainty in high-dimensional problems (like Karhunen–Loève basis fields) by accurately capturing both the structural features of the data and the known low-rank truncation limits imposed by ill-posed physical models.

  4. Handle complex, observation-dependent inference tasks (like reconstructing a 3D field from limited projections) efficiently, yielding a per-pixel uncertainty map that is calibrated to the exact posterior within its relevant subspace.

  5. Be robust across different data modalities and model structures (e.g., handling both learned flow posteriors and discrete mixture priors) by utilizing a reference-agnostic estimation mechanism.

Abstract

Uncertainty in the solution of an inverse problem and in the tasks performed on it is quantified by posterior expectations, each an average of an integrand over M posterior samples. While designed quadratures improve on the O(M-1/2) error of Monte-Carlo estimation, they solve an optimization problem, often against the posterior density, for every new observation, which can be computationally costly. To address this limitation, we introduce the quadrature field, a set-equivariant network that maps an observation and its M posterior samples to an M-node signed-weight quadrature in one forward pass. Trained once on a family of posteriors to minimize the worst-case integration error over a class of functions, it serves any observation, any M and any integrand in that class with no further optimization. We show that, with high probability and up to a computable slack, the resulting quadrature is never worse than the Monte-Carlo estimate built from the same samples. We validate the quadrature field on closed-form and on learned posteriors, one constrained by a partial differential equation, where it improves on the Monte-Carlo estimate in median at every node count, often by orders of magnitude.

Sources

Related papers