Understanding the Synergy between SFT, RLVR, and OPD in LLM Post-Training
cs.LG, cs.AI
Submitted: 2026-09-25
Updated: 2026-09-25
Terminology
Sources
- SFT Overtraining Predicts Rank Inversion via Entropy Collapse Under RLVR
- Weak-to-Strong Generalization via Direct On-Policy Distillation
- Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes
- MiniLLM: On-Policy Distillation of Large Language Models
- Reinforced Self-Training (ReST) for Language Modeling
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- A Predictive Law for On-Policy Self-Distillation From World Feedback
- Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe
- When RL Fails after SFT: Rejuvenating Model Plasticity for Robust SFT-to-RL Handoff
- KL for a KL: On-Policy Distillation with Control Variate Baseline
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Stage-1 Controls the Entropy Regime, Not the Outcome
- On the Geometry of On-Policy Distillation
- HybridFlow: A Flexible and Efficient RLHF Framework
- Demystifying On-Policy Distillation: Roles, Pathologies, and Regulations
- MiMo-V2-Flash Technical Report
- Trust Region On-Policy Distillation
- Qwen3 Technical Report
- Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillation
- GLM-5: from Vibe Coding to Agentic Engineering
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks