Contrastive Linearized Laplace: Loss Geometry, Valid Repairs, and Reliable Cores

summary

The gist

The gist The proposed framework develops a post-hoc method based on Linearized Laplace Approximation (LLA) to propagate encoder uncertainty through downstream tasks, yielding reliable cores that

In short

The framework uses Linearized Laplace Approximation (LLA) to propagate encoder uncertainty through downstream tasks. It moves beyond simple representation variance by analyzing the curvature of contrastive objectives and applying specific mathematical repairs to ensure valid local Gaussian approximations. This yields 'reliable cores' that quantify decision stability rather than just embedding variance.

Key concepts

Linearized Laplace Approximation (LLA): LLA is a method used to approximate complex probability distributions, like those from contrastive losses, as a simple Gaussian distribution around a specific point. It involves linearizing the objective function around that point to simplify the calculation of uncertainty without needing complex retraining.
Representation Uncertainty vs. Decision Uncertainty: Representation uncertainty refers to how much the encoder's output embedding varies across different inputs. Decision uncertainty, in contrast, measures how stable a specific downstream task outcome (like retrieval or clustering) is when those embeddings are slightly perturbed. The paper argues that representation variance is insufficient on its own.
Post-hoc Framework and PSD Repairs: This is a method applied *after* the encoder is trained to propagate uncertainty through the task. It involves characterizing the curvature of contrastive objectives and introducing 'positive-semidefinite (PSD) repairs' to fix mathematical issues that arise in local Gaussian approximations, ensuring the resulting uncertainty estimates are valid.

Terminology used across episodes

Reinforcement Learning (RL) — explained in 25 episodes
Sample Complexity — explained in 20 episodes
Retrieval-Augmented Generation (RAG) — explained in 18 episodes
Federated Learning — explained in 17 episodes
Quantum Fisher Information (QFI) — explained in 16 episodes
Flow Matching — explained in 15 episodes

This episode discusses

The paper

Contrastive Linearized Laplace: Loss Geometry, Valid Repairs, and Reliable Cores · Read on arXiv

F. J. Saez-Maldonado, Luis A. Ortega, Andres R. Masegosa, Thomas D. Nielsen

University of Granada · Aalborg University

While contrastive learning is widely used to learn deterministic representations, it does not inherently provide estimates of uncertainty. Post-hoc uncertainty estimation methods, such as the Linearized Laplace approximation (LLA), offer a way to quantify uncertainty without retraining the encoder. However, we show that directly using the LLA induced distribution over the contrastive embedding does not yield reliable pointwise uncertainty estimates for downstream tasks: embedding variance alone does not reflect whether decisions such as nearest-neighbor retrieval or cluster assignment are stable. To address this limitation, we develop a post-hoc framework based on LLA that propagates encoder uncertainty through the downstream task. We characterize the curvature of contrastive objectives and introduce positive-semidefinite repairs where needed to obtain valid local Gaussian approximations without retraining the encoder. We test the proposed approach in metagenomic binning and image classification tasks. For metagenomic binning, posterior coassignment probabilities support consensus partitions and the selection of reliable cores, leading to improvements over existing baselines. Additional image classification experiments show calibration improvements on standard benchmarks and real histopathology datasets.

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Contrastive Linearized Laplace".

Jane: The gist The proposed framework develops a post-hoc method based on Linearized Laplace Approximation (LLA) to propagate encoder uncertainty through downstream tasks,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: So, to get into the nitty-gritty of "Contrastive Linearized Laplace: Loss Geometry, Valid Repairs, and Reliable Cores," they’re focusing on how to make that uncertainty estimation method actually work reliably for real tasks. They are looking at the geometry of the contrastive loss functions.

Jane: That sounds a bit abstract for someone who isn't deep in variational inference. What does "loss geometry" mean in practice here? Are we talking about something complicated?

Lu: Well, they show that the validity of this uncertainty estimate depends directly on the curvature of the contrastive loss with respect to its relational scores. They characterize this curvature and then introduce some positive-semidefinite repairs when they need them.

Meng: So they’re patching things up mathematically so that their local Gaussian approximations are actually valid, even when the loss functions behave weirdly, like distance or margin losses that might have negative curvature.

Lalam: They address this by distinguishing representation uncertainty from decision uncertainty and then propagating the encoder uncertainty through the downstream rule to measure decision stability instead of just looking at embedding variance alone.

The paper's summary: Tom: So, to sum up what they did in "Contrastive Linearized Laplace: Loss Geometry, Valid Repairs, and Reliable Cores," they developed a way to propagate uncertainty through the downstream task by characterizing the curvature of contrastive objectives and introducing positive-semidefinite repairs when necessary.

Jane: That’s a big step because it means we can get more reliable cores—which they define as subsets of data whose assignments remain stable under posterior perturbations—instead of just using some simple variance score.

Lu: They show that this propagation through the downstream rule actually captures whether the resulting groups or decisions are stable when you perturb the encoder. This yields these reliable cores summarized through quality–coverage tradeoffs.

Meng: So, it’s not about finding a single perfect uncertainty number; it’s about summarizing uncertainty based on a specific downstream rule, which is what matters for deployment.

Lalam: They conclude that reliable cores are task-specific summaries of uncertainty under a particular downstream rule and posterior rather than universal measures of how uncertain the encoder is.

The paper's improvements: Tom: Now, they highlight a few specific improvements in this work. First, they formulate a relational LLA for pairwise and candidate-set contrastive losses and characterize posterior validity through the output-space geometry of these objectives.

Jane: That sounds like they’re giving specific mathematical conditions for when their method actually gives you a correct result. They show that logit-Bernoulli and InfoNCE objectives have positive-semidefinite output, which is good because it makes the resulting local posterior well defined.

Lu: For distance-based objectives like distance-Bernoulli and margin losses, they introduce positive-semidefinite repairs to make the resulting local posterior well defined without retraining the encoder. They use things like spectral repair or radial-only repair for distance losses to keep only the radial curvature.

Meng: So they’re making sure that even if a loss function is tricky, like those distance losses with negative curvature, their math stays sound because they are projecting indefinite blocks onto the positive semi-definite cone.

Lalam: They also mention an alternative called empirical Fisher which replaces second-order curvature with first-order sensitivity at the MAP solution, giving a PSD block by construction without needing Hessian eigendecompositions.

Conclusion: Tom: So, to wrap up "Contrastive Linearized Laplace: Loss Geometry, Valid Repairs, and Reliable Cores," the main implication is that we move away from just looking at representation variance and toward a method that explicitly measures decision stability through propagation through the downstream rule.

Jane: That’s the big shift—from representation uncertainty to decision uncertainty—and they provide tools like those PSD repairs to make it mathematically sound when dealing with different types of contrastive losses.

Lu: The results show that posterior agreement after applying the clustering rule directly captures whether the resulting groups remain stable under encoder perturbations, which is a much more useful summary than a simple marginal or geometric uncertainty score.

Meng: For production, this means we can select robust subsets of data—those reliable cores—based on their stability under the actual downstream rule, not just an arbitrary threshold.

Lalam: And they confirm that this approach yields state-of-the-art performance on genome clustering benchmarks and shows that posterior averaging is not just filtering ambiguous parts, but actually stabilizing the recovered bin structure in metagenomic binning.

Tom: So, "Contrastive Linearized Laplace: Loss Geometry, Valid Repairs, and Reliable Cores" gives us a framework to get stable cores by tying uncertainty directly to the decision rule itself. That’s what it boils down to.

More episodes

← Home