Agentic Reinforcement Learning with Observation-Calibrated Self-Distillation
Yi Yang, Cong Qin, Xiaodan Liu, Chishui Chen, Qing Dong, Yan Zhang, Cao Liu, Zhao Yang, Lu Pan, Jiaye Lin, Yi Feng
cs.LG, cs.AI, cs.CL
Submitted: 2026-08-05
Code: https://github.com/yiy1x/OCSD
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Resolving Action Bottleneck: Agentic Reinforcement Learning Informed by Token-Level Energy
- On-Policy Delta Distillation
- Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning
- Rethinking On-Policy Self-Distillation for Thinking Models
- Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs?
- OpenVLA: An Open-Source Vision-Language-Action Model
- What and When to Distill: Selective Hindsight Distillation for Multi-Turn Agents
- HERO: Hindsight-Enhanced Reflection from Environment Observations for Agentic Self-Distillation
- When Are Teacher Tokens Reliable? Position-Weighted On-Policy Self-Distillation for Reasoning
- Self-Distilled Agentic Reinforcement Learning
- RLCSD: Reinforcement Learning with Contrastive On-Policy Self-Distillation
- Privileged Information Distillation for Language Models
- Diagnosing and Mitigating Thinking Collapse in On-Policy Self-Distillation
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Purified OPSD: On-Policy Self-Distillation Without Losing How to Think
- ALFWorld: Aligning Text and Embodied Environments for Interactive Learning
- PBSD: Privileged Bayesian Self-Distillation for Long-Horizon Credit Assignment
- UCOB: Learning to Utilize and Evolve Agentic Skills via Credit-Aware On-Policy Bidirectional Self-Distillation
- Denser not equal to Better: Limits of On-Policy Self-Distillation for Continual Post-Training
- Demystifying On-Policy Distillation: Roles, Pathologies, and Regulations
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks