Counteraction-Aware Multi-Teacher On-Policy Distillation for General Capability Recovery with Domain Preservation
cs.AI
Submitted: 2026-05-26
Updated: 2026-09-14
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs
- Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes
- GLM-5: from Vibe Coding to Agentic Engineering
- SelecTKD: Selective Token-Weighted Knowledge Distillation for LLMs
- Baichuan-M3: Modeling Clinical Inquiry for Reliable Medical Decision-Making
- m1: Unleash the Potential of Test-Time Scaling for Medical Reasoning with Large Language Models
- Llama-Nemotron: Efficient Reasoning Models
- LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
- An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning
- Explain in Your Own Words: Improving Reasoning via Token-Selective Dual Knowledge Distillation
- Scaling Reasoning Efficiently via Relaxed On-Policy Distillation
- Qwen3 Technical Report
- Continual Learning: Tackling Catastrophic Forgetting in Deep Neural Networks with Replay Processes
- CoSER: A Comprehensive Literary Dataset and Framework for Training and Evaluating LLM Role-Playing and Persona Simulation
- Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
- MedReason: Eliciting Factual Medical Reasoning Steps in LLMs via Knowledge Graphs
- MiMo-V2-Flash Technical Report
- LLM-Oriented Token-Adaptive Knowledge Distillation
- PACED: Distillation and On-Policy Self-Distillation at the Frontier of Student Competence
- Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillation
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
- VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection