Beyond Forgetting: Representation Misdirection Elicits Controllable Side Behaviors and Capabilities
cs.LG, cs.CL
Submitted: 2026-01-29
Updated: 2026-09-20
Comments: 49 pages, 22 tables, 27 figures
Code: https://github.com/meta-llama/llama3
License: http://creativecommons.org/licenses/by/4.0/
The gist: We consider Representation Misdirection (RM), a class of large language model (LLM) unlearning methods that achieve forgetting by redirecting the latent representations of forget-samples toward a
Terminology
Abstract
We consider Representation Misdirection (RM), a class of large language model (LLM) unlearning methods that achieve forgetting by redirecting the latent representations of forget-samples toward a target vector. Despite being important, the roles of the target vector used in RM, however, remain underexplored. Here, we approach and revisit RM through the lens of the Linear Representation Hypothesis. Specifically, if one can identify a one-dimensional representation corresponding to a high-level concept, the Linear Representation Hypothesis enables linear operations on this concept vector within the forget-representation space. Under this view, we hypothesize that, beyond forgetting, machine unlearning via RM elicits controllable side effect behaviors and capabilities corresponding to the high-level concept. Our hypothesis is empirically validated across a wide range of concepts and tasks, including controlling unlearned models' truthfulness, sentiment, stereotypical bias, refusal, language, and in-context learning (ICL) tasks. Our findings reveal that this phenomenon could be either a hidden risk if misused or a mechanism that can be harnessed for developing unlearned models that require stronger capabilities and controllable behaviors.
Sources
- Open Problems in Machine Unlearning for AI Safety
- Feature-Selective Representation Misdirection for Machine Unlearning
- Training Verifiers to Solve Math Word Problems
- Do Unlearning Methods Remove Information from Language Model Weights?
- Does Unlearning Truly Unlearn? A Black Box Evaluation of LLM Unlearning Methods
- Who's Harry Potter? Approximate Unlearning in LLMs
- Toy Models of Superposition
- Applying sparse autoencoders to unlearn knowledge in language models
- Studying Large Language Model Generalization with Influence Functions
- Second-Order Information Matters: Revisiting Machine Unlearning for Large Language Models
- Investigating Model Editing for Unlearning in Large Language Models
- BLUR: A Benchmark for LLM Unlearning Robust to Forget-Retain Overlap
- Mistral 7B
- Exact Unlearning of Finetuning Data via Model Merging at Scale
- Delta-Influence: Unlearning Poisons via Influence Functions
- Editing as Unlearning: Are Knowledge Editing Methods Strong Baselines for Large Language Model Unlearning?
- Representation-Aware Unlearning via Activation Signatures: From Suppression to Entity-Signature Erasure
- Dissecting Language Models: Machine Unlearning via Selective Pruning
- SoK: Machine Unlearning for Large Language Models
- Guardrail Baselines for Unlearning in LLMs
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks