Why On-Policy Distillation Sometimes Fails: Vanishing Learning Signals
cs.LG, cs.CL
Submitted: 2026-10-08
Updated: 2026-10-08
Code: https://github.com/leizhao7/opd-learning-signals
Terminology
Sources
- Unmasking On-Policy Distillation: Where It Helps, Where It Hurts, and Why
- Learning to Foresee: Unveiling the Unlocking Efficiency of On-Policy Distillation
- Process Reinforcement through Implicit Rewards
- Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes
- Rethinking On-Policy Distillation of Large Language Models II: One Training Example
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- JustRL: Scaling a 1.5B LLM with a Simple RL Recipe
- Skywork Open Reasoner 1 Technical Report
- Multi-IF: Benchmarking LLMs on Multi-Turn and Multilingual Instructions Following
- Asymmetric On-Policy Distillation: Bridging Exploitation and Imitation at the Token Level
- Cornerstones or Stumbling Blocks? Deciphering the Rock Tokens in On-Policy Distillation
- Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
- Reinforcement Learning Finetunes Small Subnetworks in Large Language Models
- PRISM: Demystifying Retention and Interaction in Mid-Training
- On the Geometry of On-Policy Distillation
- Demystifying On-Policy Distillation: Roles, Pathologies, and Regulations
- Not All Disagreement Is Learnable: Token Teachability in On-Policy Distillation
- MiMo-V2-Flash Technical Report
- Qwen3 Technical Report
- Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolation
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks