Look Ahead Before You Distill: Future Trajectory Validation of Teacher Guidance for Agentic On-Policy Distillation

arXiv:2608.01953 · cs.CL, cs.LG · Submitted 2026-08-03 · Read on arXiv

Chishui Chen, Yaoyou Fan, Te Sun, Yi Yang, Chenghao Sun, Delin Mao, Hongbo Qiao, Zuowei Zhang, Junxi Wang, Chenxing Sun, Yangen Hu, Lu Pan, Xuyang Liu, Linfeng Zhang

cs.CL, cs.LG

Submitted: 2026-08-03

Comments: 15 pages, 5 figures

Code: https://github.com/ChenChiShui/FutureBridge-OPD

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Distillation — 64× in this paper · explained in 2 episodes
• On-Policy Distillation — 27× in this paper · explained in 3 episodes
• Benchmark — 19× in this paper · explained in 2 episodes
• Ablation — 4× in this paper · explained in 1 episode
• Alignment — 4× in this paper · explained in 1 episode
• Knowledge Distillation — 3× in this paper · explained in 11 episodes
• Ablation study — 2× in this paper · explained in 1 episode
• Agentic Task — 2× in this paper · explained in 1 episode
• Agentic Tasks — 2× in this paper · explained in 1 episode
• Aggregation — 2× in this paper · explained in 1 episode
• Credit assignment — 2× in this paper · explained in 3 episodes
• Failure Modes — 2× in this paper · explained in 3 episodes
• Adam — 1× in this paper · explained in 2 episodes
• Adaptive Weighting — 1× in this paper · explained in 1 episode
• Core — 1× in this paper · explained in 2 episodes
• Fidelity — 1× in this paper · explained in 2 episodes
• Local Optimization — 1× in this paper · explained in 2 episodes
• Perplexity — 1× in this paper · explained in 3 episodes
• Regret — 1× in this paper · explained in 3 episodes
• Self-Distillation — 1× in this paper · explained in 7 episodes
• Thinking mode — 1× in this paper · explained in 2 episodes
• Two-Stage Framework — 1× in this paper · explained in 2 episodes

Sources

• Distilling the Knowledge in a Neural Network
• Bridging Reasoning Trajectories in On-Policy Distillation via Near-Future Guidance
• Respecting Self-Uncertainty in On-Policy Self-Distillation for Efficient LLM Reasoning
• Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
• Multi-Turn On-Policy Distillation with Prefix Replay
• Your Teacher Can't Help You Here: Combating Supervision Fidelity Decay in On-Policy Distillation
• APPO: Agentic Procedural Policy Optimization
• FIPO: Eliciting Deep Reasoning with Future-KL Influenced Policy Optimization
• Trust Region On-Policy Distillation
• Pass the Baton: Trajectory-Relayed On-Policy Distillation
• Credit Assignment with Resets in Language Model Reasoning
• Qwen3 Technical Report
• DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
• Prune-OPD: Efficient and Reliable On-Policy Distillation for Long-Horizon Reasoning
• ATOD: Annealed Turn-Aware On-Policy Distillation for Multi-Turn Agentic Tasks
• TCOD: Exploring Temporal Curriculum in On-Policy Distillation for Multi-turn Autonomous Agents
• ShortOPD: Recovering Pruned LLMs with Short-to-Long On-Policy Distillation
• SCOPE: Signal-Calibrated On-Policy Distillation Enhancement with Dual-Path Adaptive Weighting
• TurnOPD: Making On-Policy Distillation Turn-Aware for Efficient Long-Horizon Agent Training

Related papers

• Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
• Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
• Subliminal Steering: Stronger Encoding of Hidden Signals
• MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
• The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
• Untangling the Mechanisms of Misleading Context in Medical Question Answering