Dense Is Not Enough: Hierarchical Supervision Allocation for Long-Horizon On-Policy Distillation

arXiv:2609.33409 · cs.CL · Submitted 2026-09-27 · Read on arXiv

cs.CL

Submitted: 2026-09-27

Updated: 2026-09-27

Terminology

• Distillation — 43× in this paper · explained in 2 episodes
• On-Policy Distillation — 22× in this paper · explained in 3 episodes
• Benchmark — 8× in this paper · explained in 2 episodes
• Ablation — 6× in this paper · explained in 1 episode
• Knowledge Distillation — 5× in this paper · explained in 11 episodes
• Self-distillation — 3× in this paper · explained in 6 episodes
• Transformer — 3× in this paper · explained in 2 episodes
• Ablation study — 2× in this paper · explained in 1 episode
• Agentic Task — 2× in this paper · explained in 1 episode
• Agentic Tasks — 2× in this paper · explained in 1 episode
• GRPO — 2× in this paper · explained in 6 episodes
• Actionability — 1× in this paper · explained in 3 episodes
• Adam — 1× in this paper · explained in 2 episodes
• Adaptation — 1× in this paper · explained in 1 episode
• Alignment — 1× in this paper · explained in 1 episode
• Consistency — 1× in this paper · explained in 3 episodes
• Decode — 1× in this paper · explained in 2 episodes
• Exposure Bias — 1× in this paper · explained in 2 episodes
• Hysteresis — 1× in this paper · explained in 3 episodes
• Regret — 1× in this paper · explained in 3 episodes
• Thinking mode — 1× in this paper · explained in 2 episodes
• Verification — 1× in this paper · explained in 2 episodes

Sources

• Look Ahead Before You Distill: Future Trajectory Validation of Teacher Guidance for Agentic On-Policy Distillation
• MiniLLM: On-Policy Distillation of Large Language Models
• Distilling the Knowledge in a Neural Network
• Asymmetric On-Policy Distillation: Bridging Exploitation and Imitation at the Token Level
• Entropy-Aware On-Policy Distillation of Language Models
• Scaling Reasoning Efficiently via Relaxed On-Policy Distillation
• On-Policy Distillation with Curriculum Turn-level Guidance for Multi-turn Agents
• Filter, Then Reweight: Rethinking Optimization Granularity in On-Policy Distillation
• DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
• Self-Distillation Enables Continual Learning
• ATOD: Annealed Turn-Aware On-Policy Distillation for Multi-Turn Agentic Tasks
• TCOD: Exploring Temporal Curriculum in On-Policy Distillation for Multi-turn Autonomous Agents
• Not All Disagreement Is Learnable: Token Teachability in On-Policy Distillation
• DASH-OPD: Discrepancy-Aware Switching with Hysteresis for On-Policy Distillation
• Finding the Evidence: Discovering Decision-Supporting Tokens for On-Policy Reasoning Distillation
• On the Position Bias of On-Policy Distillation
• TIP: Token Importance in On-Policy Distillation
• Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models
• SAGE-OPD: Selective Agent-Guided Intervention for Multi-Turn On-Policy Distillation

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering