CRAFT: Forgetting-Aware Intervention-Based Adaptation for Continual Learning

summary

Video file (mp4)

The gist

The gist: CRAFT introduces a continual learning framework that avoids updating model weights by instead learning low-rank interventions on hidden representations, which unifies routing,

In short

CRAFT is a continual learning framework that avoids updating model weights by learning low-rank interventions on hidden representations. It routes new tasks to existing similar task groups based on output distribution divergence, uses a Kullback-Leibler penalty to prevent forgetting, and merges interventions for updated tasks. This approach improves performance and reduces forgetting compared to standard LoRA methods.

Key concepts

Representation Fine-Tuning
Instead of changing the main model weights, CRAFT focuses on editing hidden states within a low-rank subspace. This reframes continual learning as organizing and updating these representation-level interventions, allowing the core backbone of the LLM to remain frozen while adaptation occurs in this learned subspace.
Task Routing on Output-Distribution Divergence
Tasks are grouped by comparing their output distributions rather than using learned gates. A task joins an existing group if its output distribution is close enough to that group's prior state, based on a distance metric calculated from how much each has moved from a baseline seed.
KL-based Forgetting Control
A Kullback-Leibler (KL) penalty is used during training. This signal keeps the new task's intervention close to the established behavior of its routed group, directly controlling forgetting of earlier tasks. A small, private intervention in deep layers preserves dedicated capacity for new knowledge.

Terminology used across episodes

This episode discusses

The paper

CRAFT: Forgetting-Aware Intervention-Based Adaptation for Continual Learning · Read on arXiv

Md Anwar Hossen, Fatema Siddika, Juan Pablo Muñoz, Tanya Roosta, Ali Jannesari

Iowa State University · Maro Systems · University of California, Berkeley

Large language models (LLMs) can learn new tasks through fine-tuning, but learning tasks sequentially often leads to catastrophic forgetting of earlier ones. This paper proposes CRAFT, a continual learning framework that keeps model weights frozen and instead learns low-rank interventions on hidden representations. Similar tasks share an intervention to help knowledge transfer, with similarity judged by how each task alters the model's output distribution rather than by a learned gating network. Within a shared intervention, a Kullback-Leibler (KL) penalty keeps each new task close to the intervention's existing behavior, which limits forgetting of earlier tasks, while a small, task-private intervention in the deepest layers preserves dedicated capacity that later tasks cannot overwrite. The learned interventions also make it possible to identify the task at test time, so task identity labels are not needed. Across multiple benchmarks and model scales, CRAFT improves overall performance and reduces forgetting compared with strong LoRA-based methods, while remaining robust to task ordering.

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "CRAFT: Forgetting-Aware Intervention-Based Adaptation for Continual Learning".

Jane: The gist: CRAFT introduces a continual learning framework that avoids updating model weights by instead learning low-rank interventions on hidden representations, which unifies routing, regularization,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: We've seen how CRAFT routes tasks and fine-tunes interventions, but the paper really focuses on the conceptual shift it brings to continual learning, which is what they call Continual Representation-Anchored Fine-Tuning for Continual Learning >

Jane: They are trying to answer three big questions in this area: where should adaptation live, how should tasks be assigned without a learned router, and how can we control forgetting during adaptation without hurting learning efficiency >

Lu: The paper argues that these challenges haven't been addressed jointly within a representation fine-tuning framework before >

Meng: That sounds ambitious. Most parameter-space methods focus on weight changes, but CRAFT is proposing something entirely different in how we structure the adaptation >

Tom: They are presenting this as a way to avoid the interference that happens when you try to isolate interventions or when you force unrelated tasks into one shared intervention >

Jane: Instead, they propose learning these interventions such that they can be organized, shared, and updated sequentially without catastrophic forgetting occurring across the sequence of tasks >

Lu: The authors are arguing that this shift reframes continual learning as a problem of organizing, sharing, and updating representation-level interventions rather than just parameter changes >

Tom: It’s a very clean way to think about it because instead of worrying about every weight in the model, you focus on managing these specific edits within a low-rank subspace >

Jane: So the implication for us is that we can tackle continual learning by focusing our attention on how these representations evolve across tasks, which seems like a more manageable problem than just trying to fine-tune everything at once >

Lu: And they show that this approach works when you use this single KL-based objective to manage routing, regularization, and merging all together >

Meng: If we can prove that organizing these interventions is the right way to do continual learning, then it opens up a whole new direction for building robust AI systems >

Tom: It’s certainly interesting because they are showing that you don't need to change the fundamental architecture of how adaptation happens; you just need a better way to organize the components >

Jane: So, after seeing this summary, what’s the next piece of the puzzle—how does this actual three-stage process work in practice when a new task arrives?

The paper's summary: Tom: Okay, so we know it routes based on output distribution divergence, then fine-tunes with KL divergence against a prior state, and finally merges the intervention using that same KL signal >

Jane: That’s the cycle of how a new task gets integrated into the system. It starts with a warm-up to get an initial intervention phi e of that new task >

Lu: During that warm-up, they compare this provisional intervention's output distribution against each existing group using symmetric KL divergence to see which one it’s closest to >

Meng: So if the distance falls below a threshold delta, the task joins the closest group; otherwise, a new group is opened for it >

Tom: That’s how they handle routing deterministically without needing a learned router parameter or gate that gets in the way of adaptation >

Jane: Then, once it finds a home, they freeze that group's state as an anchor intervention phi anchor >

Lu: And then the live intervention for the new task is trained using a forward KL penalty against this anchor to keep it close to what the group already encodes >

Tom: That KL penalty acts directly on forgetting rather than on some proxy, which is a crucial distinction for controlling training drift during adaptation >

Jane: And finally, once that new intervention phi t is trained, it overwrites the shared tiers of the existing group's state, completing the cycle before the next task arrives >

Meng: So it’s not just about adding a new piece of data; it’s about systematically updating and merging that piece into the existing structure using this KL signal >

Lu: It’s a unified objective that manages routing, regularization, and merging all through one mechanism based on output distribution KL divergence >

Tom: It sounds like they've successfully managed to keep the model stable by ensuring every adaptation step is guided by a mechanism that respects the knowledge already present in the system >

Jane: So it sounds like the main idea is that this framework systematically updates and merges existing knowledge rather than just adding new data points on top of it >

The paper's improvements: Tom: The authors highlight a few specific improvements they made to make this framework more robust, focusing heavily on how they handle those three stages we discussed >

Jane: One big improvement is the deterministic routing itself. They showed that by comparing the warmed-up new task distribution against each existing group’s stored distribution, you don't get spuriously close results if two unmoved models appear close to each other >

Lu: That means the routing mechanism is stable, even when you change parameters like delta or Swu, as long as you stay within certain ranges >

Meng: I mean if the routing threshold delta gets too small, the system might start opening too many new groups and losing track of things, which is a real deployment concern >

Tom: The paper tested this stability under those kinds of perturbations and found that the discovered cluster structure is actually invariant even when you mess with parameters by forty percent on delta and four times on Swu >

Jane: That’s a strong result because it means the way tasks are grouped is robust to those kinds of parameter changes, which suggests a very solid system >

Lu: It also confirmed that OP and BWT metrics, operational performance and backward transfer are jointly near-optimal across this joint plateau, meaning you get good performance without having to obsess over every single metric individually >

Tom: So the authors aren't just showing us a specific result; they’re showing that this design is inherently stable in its operation >

Jane: And what about forgetting specifically? They linked the regularization objective to the empirical law of Shenfeld et al. <ref:2605.05732#pg1> which shows that post-finetuning forgetting on prior tasks is directly related to the KL divergence between those policies >

Lu: So they are using this known relationship empirically to guide their regularization, ensuring that the policy stays close to what it already knows, which is a solid foundation >

Meng: That’s important because it means we aren't just guessing how much forgetting will happen; we have a theoretical reason why the system should be stable under these conditions >

Tom: And they managed to do all of this while using less trainable parameters than other methods, which speaks to the efficiency of their approach >

Jane: So, putting it all together, CRAFT is a framework that manages routing, regularization, and merging through a single KL-based objective that controls the entire adaptation pipeline >

Conclusion: Tom: To wrap up this discussion on CRAFT: we’ve seen how this framework systematically routes tasks based on output distribution divergence, fine-tunes with KL divergence against prior states, and merges interventions using that same signal >

Jane: The main implication is that continual learning can be done by focusing on organizing and updating representation-level interventions instead of just trying to update the whole model's parameters >

Lu: They have shown a lot about stability under stress tests, proving the discovered cluster structure remains stable even when you perturb routing parameters significantly >

Meng: For deployment, they’ve made it efficient enough that we can actually keep running these models without worrying about catastrophic forgetting happening constantly >

Lalam: I’m pretty sure this framework is the most promising direction because it directly addresses the core challenge of managing forgetting in a structured way rather than just patching symptoms >

Tom: It sounds like CRAFT is a solid piece of research that shows how to keep model performance high while keeping backward transfer low >

Jane: So, we’ve explored the CRAFT paper, and it’s clear that focusing on representation-level interventions offers a structured way forward for handling continual learning challenges >

More episodes

← Home