HEAL: Hindsight Entropy-Assisted Learning for Reasoning Distillation
cs.AI, cs.LG
Submitted: 2026-03-11
Updated: 2026-09-01
Comments: EMNLP 2026 Findings
Code: https://github.com/modelscope/ms-swifthttps:
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Why Distillation can Outperform Zero-RL: The Role of Flexible Reasoning
- Reinforcement Learning via Self-Distillation
- LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
- Reinforcement Learning vs. Distillation: Understanding Accuracy and Capability in LLM Reasoning
- QuestA: Expanding Reasoning Capacity in LLMs via Question Augmentation
- Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models
- FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning
- RAVR: Reference-Answer-guided Variational Reasoning for Large Language Models
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- GHPO: Adaptive Guidance for Stable and Efficient LLM Reinforcement Learning
- Self-Distillation Enables Continual Learning
- Qwen3 Technical Report
- Qwen2 Technical Report
- LIMO: Less is More for Reasoning
- Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?
- EDGE-GRPO: Entropy-Driven GRPO with Guided Error Correction for Advantage Diversity
- Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
- VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection