Formalizing Forgetting: Theory of Forgetting for Online Continual Learning

summary

The gist

The gist The authors analyze single-pass Shuffle SGD under strongly convex and general convex objectives to derive explicit upper bounds on the expected final forgetting in online continual learning,

In short

The authors analyzed single-pass Shuffle SGD under strongly convex and general convex objectives to find explicit upper bounds on expected forgetting in online continual learning. They showed that data amount, task similarity, and task ordering govern forgetting differently depending on the objective class. Empirical tests confirmed these theoretical trends in practice.

Key concepts

Propagation Factor (q): This factor describes how the error bound changes between tasks. For strongly convex objectives, increasing the per-task data amount (n) causes this factor to decrease exponentially, meaning more data helps reduce forgetting over time.
Task Similarity ($\Delta_{i,j}$): This measures how similar two learned tasks are by calculating the squared distance between their optimal models. The analysis shows that a larger gap (less similarity) never decreases the upper bound on forgetting, although task ordering influences this effect.
Task Ordering Coefficients ($c_{i,j}, g_{i,j}$): These coefficients quantify how the sequence in which tasks are learned affects the forgetting bounds. They dictate whether spacing similar tasks apart is beneficial or detrimental to minimizing the overall expected forgetting.

Terminology used across episodes

Reinforcement Learning (RL) — explained in 25 episodes
Sample Complexity — explained in 20 episodes
Retrieval-Augmented Generation (RAG) — explained in 18 episodes
Federated Learning — explained in 17 episodes
Quantum Fisher Information (QFI) — explained in 16 episodes
Flow Matching — explained in 15 episodes

This episode discusses

The paper

Formalizing Forgetting: Theory of Forgetting for Online Continual Learning · Read on arXiv

Renjie Cao, Yifei Cheng, Li Shen

Boston College · Sun Yat-sen University

Online continual learning (OCL) requires a learner to acquire a sequence of tasks in a single pass over the data, and catastrophic forgetting of earlier tasks remains its central challenge. Despite the many methods proposed to mitigate forgetting, the theoretical understanding of forgetting in OCL is still limited: existing analyses mostly focus on linear, overparameterized, or neural tangent kernel regimes, and it remains unclear which factors govern forgetting and how they affect it. To fill this gap, we analyze single-pass Shuffle SGD under strongly convex and general convex objectives and derive explicit upper bounds on the expected final forgetting. Further analysis of these bounds shows how the per-task data amount, task similarity, and task ordering enter them, and that their effects can differ across objective classes. Moreover, experiments with neural networks on Split-CIFAR-100 and PMNIST show that the trends suggested by our theory also appear in practice, indicating that these insights extend beyond the convex setting.

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Formalizing Forgetting: Theory of Forgetting for Online Continual Learning".

Jane: The gist The authors analyze single-pass Shuffle SGD under strongly convex and general convex objectives to derive explicit upper bounds on the expected final forgetting in online continual learning,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So wrapping up the discussion on "Formalizing Forgetting: Theory of Forgetting for Online Continual Learning", the authors have given us explicit upper bounds on expected forgetting under both strongly convex and general convex objectives.

Jane: That’s the main contribution, showing how data amount, task similarity, and task ordering interact to shape those bounds across different objective classes.

Lu: They explicitly show that the effects of these factors depend on whether the propagation factor is below one or not, which dictates whether spacing similar tasks apart is favored or clustering them in their bounds.

Meng: It means we have a theoretical foundation for understanding the dynamics of forgetting that’s more general than what most existing analyses cover.

Tom: Exactly. And they back it up with experiments on Split-CIFAR-one hundred and PMNIST, showing those trends appear in practice, which is pretty strong evidence <ref:2610.14787#pg1,on Split-CIFAR-100 and PMNIST>.

Jane: So what does this mean for the future of online continual learning research? The authors suggest these identified factors offer useful guidance for understanding OCL behavior.

Lu: They also point to a few areas for future work, like extending the analysis to structured non-convex settings and turning those order-dependent coefficients into practical scheduling algorithms.

Meng: I'm thinking about those scheduling algorithms—that would be really interesting if we can actually implement something based on these bounds.

Tom: That’s a natural next step for applying this work to real-world deployment challenges where you need to manage model updates dynamically.

Conclusion: Tom: So we've looked at the math on this paper, and now we're wrapping up where they are going with "Formalizing Forgetting: Theory of Forgetting for Online Continual Learning."

Jane: It really boils down to them building explicit upper bounds on how much a model forgets when it learns new tasks one pass at a time.

Lu: They used single-pass Shuffle SGD under both strongly convex and general convex objectives to get those bounds.

Meng: So, what does that actually tell us about deployment? Is this just another theoretical exercise for people sitting in offices?

Tom: Well, it shows how the data amount per task, the similarity between tasks, and the order in which they appear all enter these mathematical bounds.

Jane: Exactly. For instance, a larger gap between two tasks doesn't necessarily make forgetting worse; its effect depends on where those two tasks are in the sequence.

Lu: The results show that when the propagation factor is less than one for strongly convex objectives, spacing similar tasks apart actually helps minimize that upper bound.

Tom: That’s a specific mechanism there. It's not just "similarity matters," it's how similarity couples with ordering in a certain mathematical way.

Jane: And they proved these trends aren't just theory; they show up in experiments on Split-CIFAR-one hundred and PMNIST neural networks.

Meng: Experiments are one thing, but for me, I need to know what this means for running a model that actually breaks at scale.

Tom: It suggests that these factors give us a concrete framework to guide how we might want to schedule tasks in an online learning system.

Jane: That’s the big picture—using these bounds as guidance for designing better continual learning strategies.

Lu: The authors also pointed out that their work is currently limited to convex settings, so extending it to non-convex situations is the next natural step.

More episodes

← Home