Who Teaches Which Token? Verifier-Gated Multi-Expert On-Policy Distillation for Scientific Reasoning

arXiv:2609.15404 · cs.AI · Submitted 2026-09-14 · Read on arXiv

cs.AI

Submitted: 2026-09-14

Updated: 2026-09-15

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Distillation — 80× in this paper · explained in 2 episodes
• GRPO — 68× in this paper · explained in 6 episodes
• Verifier — 41× in this paper · explained in 2 episodes
• On-Policy Distillation — 25× in this paper · explained in 3 episodes
• Benchmark — 20× in this paper · explained in 2 episodes
• Accuracy — 9× in this paper · explained in 1 episode
• Self-distillation — 7× in this paper · explained in 6 episodes
• Verification — 7× in this paper · explained in 2 episodes
• Ablation — 6× in this paper · explained in 1 episode
• Reinforcement Learning — 4× in this paper · explained in 4 episodes
• Abstention — 3× in this paper · explained in 1 episode
• Generative AI — 2× in this paper · explained in 2 episodes
• LoRA Adapters — 2× in this paper · explained in 3 episodes
• Thinking mode — 2× in this paper · explained in 2 episodes
• Trace — 2× in this paper · explained in 2 episodes
• Aggregation — 1× in this paper · explained in 1 episode
• Calibration — 1× in this paper · explained in 7 episodes
• Consistency — 1× in this paper · explained in 3 episodes
• Entropy Collapse — 1× in this paper · explained in 2 episodes
• GRPO Objective — 1× in this paper · explained in 2 episodes
• LLM judge — 1× in this paper · explained in 2 episodes
• Regularization — 1× in this paper · explained in 3 episodes
• Reinforcement Learning (RL) — 1× in this paper · explained in 25 episodes

Sources

• On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes
• Reward-Gated On-Policy Distillation
• Rubrics as Privileged Information for Open-Ended Generation
• Counteraction-Aware Multi-Teacher On-Policy Distillation for General Capability Recovery with Domain Preservation
• Improving Data and Reward Design for Scientific Reasoning in Large Language Models
• MegaScience: Pushing the Frontiers of Post-Training Datasets for Science Reasoning
• Rubric-based On-policy Distillation
• Rethinking Reward Supervision: Rubric-Conditioned Self-Distillation
• EvoRubric: Self-Evolving Rubric-Driven RL for Open-Ended Generation
• Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains
• Distill Where You Fail: Recovering Learning Signals of Negative RL-Groups from Adaptive Teacher Guidance
• CROP: Task Relevance via Counterfactuals for Selective On-Policy Distillation
• ARES: Automated Rubric Synthesis for Scalable LLM Reinforcement Learning
• Beyond the Best Teacher: Expanding and Compressing the Reasoning Solution Manifold
• A Recipe for Long-Context Reasoning in Large Language Models via On-Policy Optimization and Distillation
• When Top-K Misses the Decision: Tool-Call Drift in Multi-Teacher On-Policy Distillation
• Woodpecker Distillation: Weak Models Diagnose Reasoning Bugs in Strong Models
• Not All Disagreement Is Learnable: Token Teachability in On-Policy Distillation
• DAPD: Dual-Anchored Policy Distillation
• CriPO: Enhancing Rubric-based RL via Self-Distillation

Related papers

• MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
• Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
• The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
• MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
• Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
• VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection