Selective Prediction Forgetting for Adaptation of Weakly Supervised Localization of Histology Images

summary

Video file (mp4)

The gist

The gist The proposed method, SFDA-DeP, introduces a Source-Free Domain Adaptation (SFDA) approach that debiases predictions for Weakly Supervised Object Localization (WSOL) in histopathology by

In short

The method SFDA-DeP addresses prediction bias in weakly supervised object localization (WSOL) during domain adaptation for histopathology images, where cross-domain shifts occur. It iteratively identifies and corrects this bias by selectively penalizing predictions for dominant classes while retaining uncertain samples. This process refines decision boundaries and improves both classification accuracy and pixel-level localization under severe distribution shifts.

Key concepts

Weakly Supervised Object Localization (WSOL)
This technique allows models to perform two tasks simultaneously: classifying the entire image based on a single class label, and precisely locating specific objects within that image. In histology, this means identifying what kind of tissue is present and where it is located in the slide.
Prediction Bias
When adapting models to new data (like different organs or staining methods), the model often becomes overly confident in a few classes—the 'dominant' ones—and ignores others. This creates a skewed distribution where most target samples are incorrectly labeled as these dominant classes, which degrades performance.
SFDA-DeP Method
This is an iterative source-free domain adaptation approach that fixes the bias. It uses two sets: a 'retain set' for confident predictions and a 'forget set' for uncertain ones from dominant classes. By adjusting the loss functions, the model learns to reduce its reliance on these biased predictions while preserving accurate localization.
Pixel-Level Classifier Optimization
Instead of just adapting the main image classifier, this method also optimizes a separate classifier at the pixel level. This helps restore fine-grained visual cues necessary for accurate object localization, ensuring that even after domain shift adaptation, the model can precisely pinpoint where objects are.

Terminology used across episodes

This episode discusses

The paper

Selective Prediction Forgetting for Adaptation of Weakly Supervised Localization of Histology Images · Read on arXiv

Alexis Guichemerre, Banafsheh Karimian, Soufiane Belharbi, Natacha Gillet, Nicolas Thome, Pourya Shamsolmoali, Mohammadhadi Shateri, Luke McCaffrey, & Eric Granger

ÉTS Montréal Institute of Technology Montreal Canada University of York McGill University Sorbonne Université

Under strong cross-domain shifts, weakly supervised object localization (WSOL) models often lose their capacity to discriminate and develop a strong prediction bias toward a subset of classes. As a result, the majority of target samples are assigned to these dominant classes. Since source-free (unsupervised) domain adaptation (SFDA) methods typically rely on pseudo-labels, inaccurate pseudo-label estimation causes suboptimal adaptation and poor classification and localization performance. This issue is particularly challenging in computational histopathology, where shifts in organ-specific morphology, staining protocols, and acquisition conditions can significantly impact class-discriminative visual cues. This paper introduces SFDA with selective prediction forgetting (SFDA-SPF), a method designed to mitigate excessive concentration of predictions for dominant classes during target-domain adaptation. SFDA-SPF identifies prediction-dominant classes based on the model's target predictions and selectively revises uncertain samples assigned to them. Importantly, this procedure does not require knowledge of the true target class distribution. In contrast with previous SFDA methods, SFDA-SPF introduces a 'forget set' composed of uncertain samples from dominant classes near their decision boundaries, enabling the classification boundaries to shift away from biased predictions. A second 'retain set' is used to anchor certain and accurate samples. To forget the predictions of dominant classes, SFDA-SPF introduces a complementary optimization objective that reduces their probability while allowing alternative class assignments to emerge. This iterative optimization process reassesses target images for forget and retain sets at regular intervals. In addition to improving image-level pseudo-label estimation, the pixel-level classifier is jointly optimized, further enhancing localization.

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: Today's paper: "Selective Prediction Forgetting for Adaptation of Weakly Supervised Localization of Histology Images".

Jane: The gist The proposed method, SFDA-DeP,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: The title itself, "Selective Prediction Forgetting for Adaptation of Weakly Supervised Localization of Histology Images," tells you exactly what they are doing: selecting which predictions to forget during adaptation to fix the domain shift issues in WSOL models.

Jane: It's quite technical, but the core idea is that they aren't just trying to adapt the model generally; they are specifically targeting and correcting prediction bias under those cross-domain shifts, which is a big step since those shifts are often really complex in histopathology >

Lu: The authors list several researchers from different universities, which suggests this work is coming from a broad collaboration tackling the problem from multiple angles, especially regarding representation learning and how features shift between domains >

Meng: It’s interesting to see the team structure here; it shows they are coming from different research backgrounds, which usually means they're looking at the problem in many ways, not just one narrow angle >

Lalam: I read their introduction, and it sets up the idea that when you have strong shifts, the model starts over-predicting some classes and under-representing others, leading to this highly skewed distribution in the target domain >

Tom: Exactly. So what they are proposing is a mechanism to actively reduce that skewed distribution by selectively forgetting predictions from those overpredicted classes during adaptation >

The paper's summary: Jane: So, what is the actual method they propose? The summary says they formulate this as an iterative process of identifying and correcting prediction bias, inspired by machine unlearning approaches >

Lu: It suggests that instead of just retraining everything from scratch or relying on simple self-training which just reinforces the initial bias, they periodically identify target images from overpredicted classes >

Meng: And for those specific images, they selectively reduce the predictive confidence for uncertain, high-entropy predictions while keeping the confident ones intact >

Tom: That sounds like a targeted way to steer the model away from those dominant class biases without throwing away all the useful information learned during training >

Lalam: The paper introduces a joint optimization of several loss terms. They have Lretain, which keeps predictions consistent with what was already predicted, and Lforget, which pushes the model to refrain from predicting that same dominant class for samples in a specific forget set >

Jane: And then there's the localization part, Lloc, which is trained separately using those pseudo-labels to anchor the localization features against the domain shift >

The paper's improvements: Tom: The key improvement seems to be this dynamic resampling of Br, which prevents early incorrect forgetting decisions from becoming irreversible by acting like an iterative correction mechanism >

Lu: That dynamic resampling is crucial because it means the model isn't just making one big decision about forgetting; it’s constantly refining its decision boundaries by reassignment >

Meng: From a deployment standpoint, that sounds robust. If we have to adapt this model on new data, we don't want one bad batch of adaptation steps to lock us into a biased state >

Jane: They also use a jointly optimized pixel-level classifier that is tuned using those class activation maps, or CAMs, which restores the localization features even under the domain shift >

Tom: So they have two main levers: first, this selective forgetting based on prediction confidence and uncertainty, and second, this joint optimization of the pixel-level head to anchor localization features >

Conclusion: Jane: To wrap up, the paper argues that this approach addresses the limitation where existing SFDA methods reinforce class asymmetries instead of correcting them >

Lu: They show that by using selective prediction forgetting, they can debias predictions for WSOL in histopathology under significant cross-domain shifts and then use a joint pixel-level head to restore localization features >

Meng: It's an improvement over previous SFDA baselines because it actively corrects the bias rather than just letting the model drift into an even worse biased state >

Lalam: The authors confirm that this method consistently outperforms state-of-the-art SFDA in both classification and localization tasks on benchmarks like PixelCAM and DeepMIL >

Tom: So, in a nutshell, Selective Prediction Forgetting for Adaptation of Weakly Supervised Localization of Histology Images proposes an iterative correction mechanism using dynamic resampling to reduce prediction bias and a joint pixel-level head to restore localization accuracy when moving between histology datasets >

More episodes

← Home