Class Machine Unlearning for Complex Data via Concepts Inference and Data Poisoning

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Class Machine Unlearning for Complex Data via Concepts Inference and Data Poisoning".

Jane: The gist The proposed concept-guided poisoning unlearning framework explicitly identifies concepts that contribute most strongly to the target class or knowledge and uses them to guide the model update,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: The paper introduces this idea of concept inference and data poisoning to tackle that uncertainty, aiming for a practical way to perform machine unlearning on complex data. It sets out to identify those specific concepts that are most strongly tied to the target class or knowledge and then use them as a guide for updating the model parameters.

Jane: What they do is create this common principle for both image classifiers and large language models, which is a big step because it brings a unified approach to unlearning across different data types.

Lu: It tackles two main technical hurdles: first, concept inference itself, which has to be done differently for images versus LLMs, and second, the controlled process of concept-guided unlearning that actually modifies the model.

Meng: I wonder how feasible this is at scale; identifying concepts in complex data sounds like it could be computationally heavy if you have millions of concepts to check.

Lalam: The paper shows they handle this by having modality-specific methods: for images, they use a Post-hoc Concept Bottleneck Model, and for LLMs, they rely on aggregating attribution scores across multiple questions.

The paper's summary: Tom: For image classification, the method first uses a Post-hoc Concept Bottleneck Model to map the latent features into interpretable concept representations that show which concepts are relevant to each class. Then they localize the specific image regions where those concepts appear using vision-language similarity, and finally they build replacement samples by inserting a patch with a different concept into that localized area.

Jane: And for LLMs, the process involves eliciting target knowledge through multiple questions that cover different facts and phrasings of what we want to forget, then they apply Integrated Gradients across those responses to aggregate word-level attributions across all questions to pinpoint the semantic content consistently contributing to that target knowledge.

Lu: The core mechanism here is that concept inference identifies a specific target concept, say "antler" for a deer image, and the poisoning step replaces that evidence with a replacement concept, say "leaf," while assigning it the label for the replacement class.

Meng: So instead of just blurring parts of an image or deleting text, they are actively swapping out those concepts in the model's representation to push it away from its old behavior. That sounds like a very specific kind of modification.

Lalam: They do this by constructing poisoned training targets where the words selected by cross-question attribution are masked with mask markers and then mixed with sampled retain anchors before the fine-tuning process begins.

The paper's improvements: Tom: The authors suggest a few key improvements. For image unlearning, they move beyond just localizing at the center of an image; they use a replacement concept patch inserted precisely at the localized target region instead.

Jane: That’s smart because it means you’re not just changing one spot randomly, but targeting exactly where that specific concept evidence is showing up in the visual data.

Lu: For LLMs, they suggest focusing on highly attributed sensitive words and entity-name words in the model responses, replacing those with mask markers while preserving the rest of the response before mixing it back into training targets.

Meng: I’m interested in how they ensure this doesn't just create noise; they need to make sure this replacement concept actually steers the model toward a different outcome without completely breaking its general knowledge.

Lalam: They also focus on quantifiable evaluation, where you monitor metrics like Aglobal and Atest while tracking how the cross-entropy losses shift, which they claim shows a balance between unlearning and keeping utility high.

Conclusion: Tom: So to wrap up, this paper shows that concept-level semantic information provides a practical interface for controlling model updates, offering one unified way to apply poisoning-based unlearning to both image classifiers and LLMs.

Jane: Concept inference is the key here because it connects the forgetting target directly to the model representations that produce the unwanted behavior. It gives us a humaninterpretable basis for what we are actually modifying in the network.

Lu: The concept inference provides a more targeted basis for constructing poisoned samples, which means we aren't just blindly poisoning data; we are poisoning based on semantic evidence of the target knowledge.

Meng: From an engineering standpoint, this suggests a way to achieve specific unlearning goals with much finer control over the training signal than what was possible before.

Lalam: Overall, the work demonstrates that concept-level semantic information can provide a practical interface between unlearning requests and controlled model updates through this concept-guided poisoning unlearning framework.

Tom: That’s all for this paper on Class Machine Unlearning for Complex Data via Concepts Inference and Data Poisoning. We’ll see how these methods evolve next time.