Decoupling Knowledge and Privacy: Post-Task Self-Distillation Replay for LLM Continual Learning
cs.LG, cs.AI
Submitted: 2026-08-31
Updated: 2026-08-31
Terminology
Sources
- Yelp Dataset Challenge: Review Rating Prediction
- Not Just After One: Sleep-Inspired Replay Prevents Catastrophic Forgetting After Sequential Tasks
- Fine-Tuning Large Language Models with User-Level Differential Privacy
- Beyond Static Models: An Evolving Framework for Continual Learning in Large Language Models across Training Stages
- Scalable Extraction of Training Data from (Production) Language Models
- From Weights to Features: SAE-Guided Activation Regularization for LLM Continual Learning
- Residual SODAP: Residual Self-Organizing Domain-Adaptive Prompting with Structural Knowledge Preservation for Continual Learning
- Revisiting the Past: Data Unlearning with Model State History
- Self-Distillation Enables Continual Learning
- Canonicalized Stable-List Replay for Private Federated Continual Learning over Language-Model Embeddings
- Federated Continual Learning for Privacy-Preserving Hospital Imaging Classification
- DP-MacAdam: Differentially Private Mechanism with Adaptive Clipping and Adaptive Momentum
- Privacy Leakage via Output Label Space and Differentially Private Continual Learning
- LLaMA: Open and Efficient Foundation Language Models
- Machine Unlearning: A Comprehensive Survey
- Qwen2.5 Technical Report
- Learning What to Forget: Improving LLM Unlearning via Learned Token-Level Importance
- Forget What's Sensitive, Remember What Matters: Token-Level Differential Privacy in Memory Sculpting for Continual Learning
- Data-Free Privacy-Preserving for LLMs via Model Inversion and Selective Unlearning
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks