Distill Where You Fail: Recovering Learning Signals of Negative RL-Groups from Adaptive Teacher Guidance
Zhuowen Han, Jinwei Xiao, Zhengxi Lu, Renren Jin, Zhiyuan Yao, Yuxin Liu, Hongyan Hao, Yueqing Sun, Yu Yang, Qi GU, Xunliang Cai, Deyi Xiong
cs.CL
Submitted: 2026-08-01
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoning
- Process Reinforcement through Implicit Rewards
- Distilling the Knowledge in a Neural Network
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Asymmetric On-Policy Distillation: Bridging Exploitation and Imitation at the Token Level
- Don't Waste Mistakes: Leveraging Negative RL-Groups via Confidence Reweighting
- Scaling Reasoning Efficiently via Relaxed On-Policy Distillation
- Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes
- No Prompt Left Behind: Exploiting Zero-Variance Prompts in LLM Reinforcement Learning via Entropy-Guided Advantage Shaping
- Unifying Group-Relative and Self-Distillation Policy Optimization via Sample Routing
- Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
- Outcome-Grounded Advantage Reshaping for Fine-Grained Credit Assignment in Mathematical Reasoning
- On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectification
- KDRL: Post-Training Reasoning LLMs via Unified Knowledge Distillation and Reinforcement Learning
- TIP: Token Importance in On-Policy Distillation
- Self-Distilled Agentic Reinforcement Learning
- Qwen2.5 Technical Report
- Demystifying OPD: Length Inflation and Stabilization Strategies for Large Language Models
- Self-Distilled RLVR
- Learning What Reinforcement Learning Can't: Interleaved Online Fine-Tuning for Hardest Questions
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering