Learning Dynamics of Continual Learning: A Unified View of Data Attribution, Forgetting, and Plasticity Loss
cs.LG, cs.AI
Submitted: 2026-09-27
Updated: 2026-09-27
Project page: https://joshua-ren.github.io/learning-dynamics-cl
Terminology
Sources
- Studying Large Language Model Generalization with Influence Functions
- Reinforced Self-Training (ReST) for Language Modeling
- Can Scale Save Us From Plasticity Loss in Large Language Models?
- Alignment Dynamics in LLM Fine-Tuning
- Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
- A Theory of Generalization in Deep Learning
- Decoupled Weight Decay Regularization
- Understanding plasticity in neural networks
- In-context Learning and Induction Heads
- Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semantics
- Learning Dynamics of Deep Learning -- Force Analysis of Deep Neural Networks
- Learning to (Learn at Test Time): RNNs with Expressive Hidden States
- LESS: Selecting Influential Data for Targeted Instruction Tuning
- HyperINF: Unleashing the HyperPower of the Schulz's Method for Data Influence Estimation
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks