Robust Code RL via Faulty-Code-Driven Test case Synthesis and Dense Reward Shaping
cs.AI, cs.SE
Submitted: 2026-08-25
Updated: 2026-08-27
Comments: Accepted by EMNLP 2026
Project page: https://atcoder.jp
License: http://creativecommons.org/licenses/by-sa/4.0/
The gist: Reinforcement Learning from Verifiable Rewards (RLVR) is pivotal for enhancing LLM code generation, yet its efficacy is often hindered by insufficient test case coverage, leading to reward hacking
Terminology
Abstract
Reinforcement Learning from Verifiable Rewards (RLVR) is pivotal for enhancing LLM code generation, yet its efficacy is often hindered by insufficient test case coverage, leading to reward hacking and policy degradation. To address this, we propose RobustTests, a framework featuring a faulty-code-driven test case synthesis strategy. By leveraging "near-correct" faulty codes, RobustTests captures latent logical discrepancies and employs validator agents with behavioral feature clustering to filter invalid or redundant test cases. Additionally, a stepwise dense reward function based on pass rates is introduced to mitigate false negatives and enhance training robustness. Using this pipeline, we construct an augmented version of the CodeContests+ dataset with superior diagnostic utility. Experimental results show that RL fine-tuning of Qwen3-32B via RobustTests achieves a 3% absolute gain on LiveCodeBench, demonstrating its effectiveness in advancing LLM code generation proficiency. Codes and data are available at https://huggingface.co/datasets/sid6/RobustTests.
Sources
- Program Synthesis with Large Language Models
- CodeContests-O: Powering LLMs via Feedback-Driven Iterative Test Case Generation
- Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
- Evaluating Large Language Models Trained on Code
- Competitive Programming with Large Reasoning Models
- Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- HardTests: Synthesizing High-Quality Test Cases for LLM Coding
- LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code
- Large Language Models as Test Case Generators: Performance Evaluation and Enhancement
- Learning to Solve and Verify: A Self-Play Framework for Code and Test Generation
- Is Self-Repair a Silver Bullet for Code Generation?
- YaRN: Efficient Context Window Extension of Large Language Models
- Proximal Policy Optimization Algorithms
- Execution-based Code Generation using Deep Reinforcement Learning
- Kimi K2: Open Agentic Intelligence
- Every Step Evolves: Scaling Reinforcement Learning for Trillion-Scale Thinking Model
- CodeContests+: High-Quality Test Case Generation for Competitive Programming
- Qwen3 Technical Report
- SWE-smith: Scaling Data for Software Engineering Agents
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems
- VSAL: A Vision Solver with Adaptive Layouts for Graph Property Detection