LastOPD: Taming Collapse in Latent On-Policy Distillation
cs.LG, cs.CL
Submitted: 2026-09-23
Updated: 2026-09-23
Code: https://github.com/Muyiiiii/LastOPD
Terminology
Sources
- Training Verifiers to Solve Math Word Problems
- Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes
- Rethinking On-Policy Distillation of Large Language Models II: One Training Example
- Verbalizable Representations Form a Global Workspace in Language Models
- JustRL: Scaling a 1.5B LLM with a Simple RL Recipe
- Measuring Mathematical Problem Solving With the MATH Dataset
- On-Policy Delta Distillation
- Distilling the Knowledge in a Neural Network
- Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
- PHF: Privileged Hidden Flow for On-Policy Self-Distillation
- FitNets: Hints for Thin Deep Nets
- Qwen3 Technical Report
- From Observations to States: Latent Time Series Forecasting
- OPRD: On-Policy Representation Distillation
- Latent On-Policy Self-Distillation
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks