SPO++: Stream-Aligned Policy Optimization for Asynchronous Agentic RL
cs.AI
Submitted: 2026-08-25
Updated: 2026-08-25
Terminology
Sources
- BASIS: Batchwise Advantage Estimation from Single-Rollout Information Sharing for LLM Reasoning
- Missing Old Logits in Asynchronous Agentic RL: Semantic Mismatch and Repair Methods for Off-Policy Correction
- Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning
- VIMPO: Value-Implicit Policy Optimization for LLMs
- A-3PO: Accelerating Asynchronous LLM Training with Staleness-aware Proximal Policy Approximation
- SAPO: Single-Rollout Autoregressive Policy Optimization for Agentic Reinforcement Learning
- Stable and Efficient Single-Rollout RL for Multimodal Reasoning
- Understanding R1-Zero-Like Training: A Critical Perspective
- Best Practice Critic Optimization
- Proximal Policy Optimization Algorithms
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- ALFWorld: Aligning Text and Embodied Environments for Interactive Learning
- Maximum Likelihood Reinforcement Learning
- Single-stream Policy Optimization
- Stale but Stable: Staleness-Adaptive Trust Regions for Stabilizing Asynchronous Reinforcement Learning
- DAPO: An Open-Source LLM Reinforcement Learning System at Scale
- What's Behind PPO's Collapse in Long-CoT? Value Optimization Holds the Secret
- VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Tasks
- Balanced Aggregation: Understanding and Fixing Aggregation Bias in GRPO
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
- VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection