On-policy Distillation with Verifiable Reward
cs.LG, cs.AI
Submitted: 2026-08-25
Updated: 2026-09-06
Code: https://github.com/LeapLabTHU/OPDVR
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- H$^2$SD: Hybrid Hindsight Self-Distillation
- Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Train Long, Think Short: Curriculum Learning for Efficient Reasoning
- ThinkPrune: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learning
- Uni-OPD: Unifying On-Policy Distillation with a Dual-Perspective Recipe
- Reinforcement Learning via Self-Distillation
- OpenAI o1 System Card
- Rethinking Entropy Regularization in Large Reasoning Models
- Entropy-Aware On-Policy Distillation of Language Models
- Unifying Group-Relative and Self-Distillation Policy Optimization via Sample Routing
- Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
- Learn to Reason Efficiently with Adaptive Length-based Reward Shaping
- KL for a KL: On-Policy Distillation with Control Variate Baseline
- Entropy-Preserving Reinforcement Learning
- Proximal Policy Optimization Algorithms
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- CE-GPPO: Coordinating Entropy via Gradient-Preserving Clipping Policy Optimization in Reinforcement Learning
- Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models
- Distilled Reinforcement Learning for LLM Post-training
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks