Spurious Advantage Hidden in GRPO
cs.AI
Submitted: 2026-09-03
Updated: 2026-09-03
Terminology
Sources
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Measuring Massive Multitask Language Understanding
- Measuring Mathematical Problem Solving With the MATH Dataset
- REINFORCE++: Stabilizing Critic-Free Policy Optimization with Global Advantage Normalization
- Conditional Advantage Estimation for Reinforcement Learning in Large Reasoning Models
- MAPO: Mixed Advantage Policy Optimization
- MO-GRPO: Mitigating Reward Hacking of Group Relative Policy Optimization on Multi-Objective Problems
- Training Verifiers to Solve Math Word Problems
- Don't Waste Mistakes: Leveraging Negative RL-Groups via Confidence Reweighting
- Proximal Policy Optimization Algorithms
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- DRPO: Efficient Reasoning via Decoupled Reward Policy Optimization
- ZeroSearch: Incentivize the Search Capability of LLMs without Searching
- MathScale: Scaling Instruction Tuning for Mathematical Reasoning
- Implicit Actor Critic Coupling via a Supervised Learning Framework for RLVR
- $\lambda$-GRPO: Unifying the GRPO Frameworks with Learnable Token Preferences
- Understanding R1-Zero-Like Training: A Critical Perspective
- Quantile Advantage Estimation: Stabilizing RLVR for LLM Reasoning
- It Takes Two: Your GRPO Is Secretly DPO
- Reinforcement Learning with Verifiable Rewards: GRPO's Effective Loss, Dynamics, and Success Amplification
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
- VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection