Rollout Efficiency in Reinforcement Learning for Reasoning Large Language Models: A Taxonomy and Future Directions
cs.AI, cs.DC
Submitted: 2026-09-21
Updated: 2026-09-21
Project page: https://yifanzhang-pro.github.io/FlashREINFORCE/FlashREINFORCE.pdf
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- Libra: Efficient Resource Management for Agentic RL Post-Training
- RolloutPipe: Overlapping Pipelined Rollout and Training in Disaggregated On-Policy LLM Reinforcement Learning
- DynaResize: Runtime GPU Reallocation for Disaggregated LLM Post-Training
- Efficient Training of Large Language Models on Distributed Infrastructures: A Survey
- RollPacker: Mitigating Long-Tail Rollouts for Fast, Synchronous RL Post-Training
- ROSE: Rollout On Serving GPUs via Cooperative Elasticity for Agentic RL
- QaRL: Rollout-Aligned Quantization-Aware RL for Fast and Stable Training under Training--Inference Mismatch
- Actor-Curator: Co-adaptive Curriculum Learning via Policy-Improvement Bandits for RL Post-Training
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- AsyncFlow: An Asynchronous Streaming RL Framework for Efficient LLM Post-Training
- History Rhymes: Accelerating LLM Reinforcement Learning with RhymeRL
- Scheduling Mixed RL Rollouts Beyond Prefix Locality
- Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning
- DUET: Optimize Token-Budget Allocation for Reinforcement Learning with Verifiable Rewards
- DORA: A Scalable Asynchronous Reinforcement Learning System for Language Model Training
- Accelerating RL Post-Training Rollouts via System-Integrated Speculative Decoding
- OpenAI o1 System Card
- VCRL: Variance-based Curriculum Reinforcement Learning for Large Language Models
- Learning as Reasoning Unfolds: Progressive Rollout Allocation for Efficient Reinforcement Learning
- Rollout-Training Co-Design for Efficient LLM-Based Multi-Agent Reinforcement Learning
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
- VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection