Large Pretraining Datasets Don't Guarantee Robustness after Fine-Tuning in Image Classification

arXiv:2410.21582 · cs.CV, cs.AI · Submitted 2024-10-28 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Large Pretraining Datasets Don't Guarantee Robustness after Fine-Tuning in Image Classification".

Jane: As a fastidious and diligent researcher, I have meticulously analyzed the provided fragments (A, B,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: We’ve covered a lot about the findings in "Large Pretraining Datasets Don't Guarantee Robustness after Fine-Tuning in Image Classification," and now we need to talk about what this all means for the bigger picture. The authors, Jaedong Hwang, Brian Cheung, Zhang-Wei Hong, Akhilan Boopathy Pulkit Agrawal, Ila Fiete from MIT found that the relationship between pretraining dataset scale and post-fine-tuning robustness is not a simple straight line we used to assume.

Jane: That’s right; the paper challenges the common assumption that bigger and more diverse foundation models automatically translate into better out-of-distribution performance after adaptation. They conclude by strongly advocating for tailored fine-tuning strategies instead of relying on blanket assumptions about pretraining dataset size.

Lu: The major implication is a necessary shift in research direction; we need to move away from just scaling up the initial training data as the sole solution and toward developing better mechanisms for continual learning that actively preserve model integrity during task specialization.

Meng: From an engineering viewpoint, this means our focus needs to be on creating fine-tuning protocols that intentionally incorporate regularization methods, because simply pushing for higher performance metrics isn't enough if the resulting model is brittle.

Lalam: For the AI ecosystem, this suggests a more mature approach where we prioritize robustness during adaptation as much as we do initial training, recognizing that specialization introduces unique vulnerabilities that need specific defenses.

Tom: So, in simple terms, the paper tells us that while massive pretraining provides a strong starting point, you still have to be smart about how you fine-tune things—you can’t just throw a huge model into the mix and expect it to perform reliably everywhere.

Jane: Precisely; the crucial lesson is that combining continual learning with robust fine-tuning methods gives us effective tools for keeping models stable when facing distribution shifts in specialized tasks.

Lu: I think this opens up a lot of exciting avenues for exploring how diverse model soup architectures can work synergistically with these continual learning techniques to build highly capable and resilient systems.

Meng: I see it as a mandate for our engineering teams to integrate these robustness checks directly into the fine-tuning loop, not just treating them as optional post-training steps.

Lalam: This paper reinforces the idea that true AI progress lies in building systems that are not only smart but also reliable when they encounter the messy reality of real-world data.

Conclusion: Tom: So we’ve seen some of the technical meat of this paper, and now it's time to zoom out and talk about what all this actually means for the folks listening at home on their commutes.

Jane: Exactly, Tom; we need to connect these complex ideas back to the real world so everyone can grasp why this research is so important for anyone working with large models.

Tom: This paper, "Large Pretraining Datasets Don't Guarantee Robustness after Fine-Tuning in Image Classification," by Hwang and his team, is essentially challenging a very common belief in the AI community.

Jane: It’s about showing that just having a massive initial dataset isn't a guaranteed shield against problems when you adapt those models for specific jobs.

Lu: From my perspective, it opens up fascinating theoretical avenues because it suggests that pretraining data scale isn't the only vector determining a model's post-fine-tuning stability.

Meng: Practically speaking, what this means is we can’t just treat fine-tuning as a simple performance boost; we have to build in active defense mechanisms against distribution shifts.

Lalam: And for our LLM culture, it suggests that building resilience into the adaptation phase is just as crucial as the initial training phase for creating trustworthy systems.

Tom: Right, so the authors found that bigger starting models can sometimes actually make robustness worse when you fine-tune them on smaller data sets.

Jane: That’s a key finding; it points to a trade-off where maximizing scale doesn't automatically maximize reliability in specialized applications.

Lu: The analysis using tools like Centered Kernel Alignment showed how much the representations shift between pretraining and fine-tuning, which is super interesting conceptually.

Meng: I wonder how this plays out when we deploy these models in real-world scenarios where the data distribution might be quite different from what they saw during pretraining.

Lalam: If we can reliably predict and mitigate these shifts, it could really help build more dependable AI systems for everyday cultural applications.

Massachusetts Institute of Technology

cs.CV, cs.AI

Submitted: 2024-10-28

Updated: 2026-10-06

Project page: https://jd730.github.io/projects/ImageNet-RIB

Importance score: 92/100

The gist: As a fastidious and diligent researcher, I have meticulously analyzed the provided fragments (A, B, and C) pertaining to the paper "Large Pretraining Datasets Don't Guarantee Robustness after

Key concepts

Robustness Inheritance Benchmark (ImageNet-RIB): A new test created to measure how well models maintain their ability to handle unexpected data after being trained on a smaller, specialized dataset. It helps researchers see if the initial training scale truly translates into better real-world reliability for new tasks.
Scale vs. Robustness Trade-off: The study found that starting with the largest foundation models does not automatically lead to more robust models after fine-tuning. In fact, larger models sometimes show worse robustness degradation when adapting them to smaller, specialized datasets compared to smaller starting points.
Continual Learning Methods: Techniques used during fine-tuning that allow a model to learn from new data without completely forgetting what it learned during its initial massive pretraining. These methods help the model intelligently balance knowledge from both the old and new distributions.
Weight Averaging (MS:PRE-FT-EWC-LwF): A specific regularization technique identified as highly effective. This method helps stabilize the fine-tuning process by averaging weights, which significantly boosted robustness improvements across different model backbones tested in the study.

Terminology

Fine-Tuning — 270× in this paper · explained in 5 episodes
Robustness — 163× in this paper · explained in 5 episodes
Accuracy — 133× in this paper · explained in 1 episode
Linear Probing — 86× in this paper · explained in 2 episodes
Benchmark — 39× in this paper · explained in 2 episodes
Continual Learning — 29× in this paper · explained in 2 episodes
Catastrophic Forgetting — 22× in this paper · explained in 5 episodes
Generalization — 20× in this paper · explained in 13 episodes
Transformer — 20× in this paper · explained in 2 episodes
Alignment — 19× in this paper · explained in 1 episode
Regularization — 14× in this paper · explained in 3 episodes
Distribution Shift — 11× in this paper · explained in 5 episodes
Adaptation — 8× in this paper · explained in 1 episode
Centered Kernel Alignment — 7× in this paper
Mean Robustness Improvement — 7× in this paper
Optimal Transport — 6× in this paper · explained in 5 episodes
Foundation Model — 5× in this paper · explained in 3 episodes
Optimal Transport Dataset Distance — 5× in this paper
Robustness Inheritance Benchmark — 5× in this paper
Centered Kernel Alignment (CKA) — 4× in this paper · explained in 2 episodes
Ablation — 3× in this paper · explained in 1 episode
Ablation Studies — 3× in this paper · explained in 1 episode
Benchmarking — 3× in this paper · explained in 3 episodes
Data Augmentation — 3× in this paper · explained in 7 episodes
Normalized Compression Distance — 3× in this paper
Consistency — 2× in this paper · explained in 3 episodes
Contrastive Learning — 2× in this paper · explained in 5 episodes
Core — 2× in this paper · explained in 2 episodes
Downstream Tasks — 2× in this paper · explained in 2 episodes
Foundation Models — 2× in this paper · explained in 7 episodes
Generalizability — 2× in this paper · explained in 6 episodes
Adam — 1× in this paper · explained in 2 episodes
Clustering — 1× in this paper · explained in 2 episodes
Distillation — 1× in this paper · explained in 2 episodes
Domain Shift — 1× in this paper · explained in 4 episodes
Fidelity — 1× in this paper · explained in 2 episodes
Fisher Information — 1× in this paper · explained in 3 episodes
Knowledge Distillation — 1× in this paper · explained in 11 episodes
Linear probe — 1× in this paper · explained in 2 episodes
Meta-learning — 1× in this paper · explained in 2 episodes
Reinforcement Learning — 1× in this paper · explained in 4 episodes
Representation Learning — 1× in this paper · explained in 2 episodes
Semantic Understanding — 1× in this paper · explained in 2 episodes
Stochastic Gradient Descent (SGD) — 1× in this paper · explained in 5 episodes
Transformer Architecture — 1× in this paper · explained in 2 episodes
Vision Transformer (ViT) — 1× in this paper · explained in 2 episodes

Summary

As a fastidious and diligent researcher, I have meticulously analyzed the provided fragments (A, B, and C) pertaining to the paper Large Pretraining Datasets Don't Guarantee Robustness after Fine-Tuning in Image Classification.

The provided text is highly fragmented. Fragment A contains a detailed narrative summary of the paper's core findings and contributions, while Fragments B and C consist primarily of references to tables (Table 33, Table 51, Table 54, etc.) without the actual data or context necessary for a complete analysis.

Therefore, I must synthesize the comprehensive narrative from Fragment A with the contextual clues provided by Fragments B and C to construct a long and detailed summary.

This paper critically examines the robustness of large pre-trained models when subjected to fine-tuning for specialized downstream tasks, specifically focusing on whether the scale or diversity of the initial pretraining dataset guarantees superior out-of-distribution (OOD) generalization after adaptation. The central thesis challenges the common assumption that larger and more diverse foundation models inherently possess better robustness post-fine-tuning.

The primary motivation stems from the widespread practice of leveraging large pre-trained models as foundations for new tasks via fine-tuning. A critical, yet often overlooked, concern is robustness: the model's ability to perform reliably on OOD data. The authors observed that standard fine-tuning often leads to significant robustness degradation and catastrophic forgetting in these models.

The study systematically investigated this issue by proposing the Robustness Inheritance Benchmark (ImageNet-RIB), a novel benchmark designed for this purpose. The research yielded several profound findings:

Scale vs. Robustness Trade-off: The most surprising finding is that models pretrained on the largest and most diverse datasets (e.g., LAION-2B) often exhibit worse robustness losses and lower absolute robustness after fine-tuning on small, specialized datasets, compared to models pretrained on smaller datasets (e.g., ImageNet-1K or LAION-100M). This suggests that starting with the strongest foundation model is not an optimal strategy for specialist tasks.
Downstream Data Sensitivity: The size of the downstream fine-tuning dataset plays a significant, detrimental role in catastrophic forgetting. Insufficient data exacerbates this forgetting, particularly noted for CLIP models, which require sufficiently large fine-tuning datasets to maintain robustness against distribution shifts.
Representation Shifts: Analysis using Centered Kernel Alignment (CKA) revealed that deeper layers of the network exhibit greater discrepancies between the pretrained and fine-tuned representations, a phenomenon amplified in models pretrained on larger datasets.
Model Soup Efficacy: The combination of a model soup approach—utilizing multiple backbones—with continual learning techniques consistently achieved superior performance in terms of robustness improvement (mean robustness improvement, mRI).

To counteract the observed degradation, the authors propose and demonstrate effective mitigation strategies:

Continual Learning Methods: Regularization-based continual learning methods were shown to improve robustness by intelligently leveraging both the original pretraining distribution and the new fine-tuning distribution.
Robust Fine-Tuning Strategies: The improvements in robustness are significantly amplified when these continual learning methods are combined with robust fine-tuning approaches.
Specific Techniques: The study highlights the efficacy of specific regularization techniques, such as Weight Averaging (MS:PRE-FT-EWC-LwF), which achieved the highest or second-highest mean robustness improvement across various ImageNet-based backbones.
Model Soup Synergy: The combination of model soup architectures with continual learning techniques proved highly effective in preserving and enhancing robustness.

The overarching conclusion is that the relationship between pretraining dataset scale and post-fine-tuning robustness is complex, not monotonic. The authors strongly advocate for tailored fine-tuning strategies over blanket assumptions about pretraining dataset size. They emphasize that while large datasets provide a strong initial foundation, they can introduce vulnerabilities during specialization if not paired with appropriate regularization and continual learning techniques. The primary takeaway is that continual learning methods and robust fine-tuning approaches, when combined, are effective tools for preserving or even improving robustness in the face of distribution shifts.

*(This synthesis integrates the narrative from A with the contextual data points implied by B and C to provide a detailed, high-fidelity summary suitable for a rigorous research context.

Improvements for AI systems

As a fastidious and diligent researcher, I have analyzed the provided paper on Large Pretraining Datasets Don’t Guarantee Robustness After Fine-Tuning. The core finding is that models pretrained on massive datasets (like LAION-2B or OpenAI) exhibit severe catastrophic forgetting and loss of out-of-distribution (OOD) generalization when fine-tuned, especially if the downstream task is trained on a small dataset.

Here are the specific improvements and capabilities this research enables for AI systems:

The primary improvement lies in developing a methodology to systematically assess and mitigate catastrophic forgetting during fine-tuning, moving beyond simple fine-tune everything approaches.

- Systematic Robustness Benchmarking (ImageNet-RIB):
The paper introduces the ImageNet-RIB benchmark, which involves sequentially fine-tuning a pretrained model on one OOD task and then testing its robustness on a diverse set of remaining OOD tasks.

3. This allows researchers to quantify the exact Robustness Inheritance of a foundation model across various downstream domains, rather than just measuring performance on one task.

- Informed Pretraining Strategy:

3. The findings suggest that starting with the largest and most diverse datasets (e.g., LAION-2B) is not always optimal for specialist tasks because it leads to larger robustness losses after fine-tuning on small datasets.

4. This enables a strategy where smaller, more targeted pretraining datasets (e.g., ImageNet-1K or ImageNet-21K) might provide a stronger robust foundation for specific downstream applications, leading to better OOD generalization when combined with the right fine-tuning technique.

- Optimized Fine-Tuning Techniques:

6. The study identifies superior methods for preserving robustness during fine-tuning, specifically:

7. • Combination of regularization-based continual learning methods (like EWC or LwF) with robust fine-tuning techniques (like Model Soup, WiSE-FT, or LP-FT).

• Linear Probing (fine-tuning only the last layer) when using large pretraining models like LAION-2B.

9. • The best ratio for weight averaging in methods like WiSE-FT (identified via grid search).

- Enhanced Model Adaptation:
The research demonstrates that techniques like LoRA and Visual Prompt are less effective at learning from fine-tuning data, suggesting they should be reserved for specific adaptation tasks rather than comprehensive robustness preservation during standard fine-tuning.
- Representation Analysis (CKA):
Using Centered Kernel Alignment (CKA) analysis allows researchers to pinpoint exactly which layers of the Transformer are suffering the most catastrophic forgetting when fine-tuning, guiding targeted architectural or regularization fixes for future models.

The resulting improved AI systems can perform the following:

Robust deployment in specialized domains: An AI system fine-tuned on a smaller, robust foundation model (e.g., ImageNet-21K) will maintain high performance and generalization when deployed on a narrow, out-of-distribution dataset (like ImageNet-Sketch or ObjectNet), which is crucial for safety in constrained environments.
Reliable Zero-Shot/Few-Shot Classification: By leveraging the findings on CLIP models, systems can be fine-tuned effectively using optimized methods to maintain strong performance when adapting to new visual concepts without extensive retraining.
Adaptive Continual Learning Systems: AI agents can utilize continual learning strategies (like EWC or LwF) during sequential task learning, ensuring that knowledge gained from early tasks (e.g., ImageNet-1K) is not destroyed when learning later, more specific tasks.
Model Selection for Pretraining: A decision framework can be built to select the optimal pretraining dataset size for a given downstream task's expected data characteristics, avoiding the pitfalls of using excessively large datasets if the subsequent fine-tuning data is small or specialized.