Ockhamareto: Pareto-Gated Segment-Level Credit Assignment for Concise Unit-Test Generation with Reinforcement Learning

arXiv:2608.24473 · cs.SE, cs.CL · Submitted 2026-08-25 · Read on arXiv

cs.SE, cs.CL

Submitted: 2026-08-25

Updated: 2026-08-25

Code: https://github.com/bytedance/SandboxFusion

License: http://creativecommons.org/licenses/by/4.0/

The gist: We introduce Ockhamareto, a single-shot GRPO framework for unit-test generation and selection, based on the principles of Ockham's Razor and Pareto Optimality.

Terminology

Abstract

We introduce Ockhamareto, a single-shot GRPO framework for unit-test generation and selection, based on the principles of Ockham's Razor and Pareto Optimality. Ockhamareto has two principal components: (i) a Pareto-gated Bonus that rewards only rollouts non-dominated in (mutation, - #tests) space, and (ii) Token-level Segment Credit, which attributes each test's marginal mutation kills back to the tokens of its unit-test block. On the UnLeakedTestBench (ULT), Ockhamareto strictly Pareto-dominates the strongest RL baseline (MIST-RL). Furthermore, it dominates on each and all optimization objectives, catching more bugs (49.9% vs 31.3% mutation score at N = 5), using fewer tests (2.60 vs 4.67 on average), thereby achieving 3.4 times the per-test trade-off improvement. The advantage is found in all four benchmarks (HumanEval+, MBPP+, CodeContests, TestGenEval-Lite): Ockhamareto leads both mutation and coverage metrics on every one, always with the smallest suite. Ockhamareto also outperforms the state-of-the-art at all model scales, adding +30 -- 35 pp mutation at 4B, 9B, and 27B model sizes. We also show that the knee point of the optimal trade-off between efficiency and effectiveness on the Pareto front is not correlated with obvious more easily computed proxy metrics, such as function size. This finding motivates the Pareto front computation; it is needed to identify this crucial engineering trade-off for each function under test.

Sources

Related papers