Measuring and Mitigating Solution Mode Collapse in RLVR
cs.LG, cs.CL
Submitted: 2026-10-08
Updated: 2026-10-08
Terminology
Sources
- How LLMs Distort Our Written Language
- SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model
- Large Language Monkeys: Scaling Inference Compute with Repeated Sampling
- Pass@k Training for Adaptively Balancing Exploration and Exploitation of Large Reasoning Models
- Training Verifiers to Solve Math Word Problems
- The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models
- RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment
- Argument Collapse: LLMs Flatten Long-Form Public Debate
- SetPO: Set-Level Policy Optimization for Diversity-Preserving LLM Reasoning
- The Choice of Divergence: A Neglected Key to Mitigating Diversity Collapse in Reinforcement Learning with Verifiable Reward
- Beyond Mode Collapse: Distribution Matching for Diverse Reasoning
- Understanding R1-Zero-Like Training: A Critical Perspective
- Uniform-Correct Policy Optimization: Breaking RLVR's Indifference to Diversity
- Playing Atari with Deep Reinforcement Learning
- Illuminating search spaces by mapping elites
- Qwen2.5 Technical Report
- Experience Replay for Continual Learning
- Expected Return Causes Outcome-Level Mode Collapse in Reinforcement Learning and How to Fix It with Inverse Probability Scaling
- Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
- Outcome-based Exploration for LLM Reasoning
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks