Rubrics on Trial: Evolving Rubrics from a Single Query via Synthetic Pairwise Evidence

Rubrics provide structured signals for training and evaluating large language models (LLMs), but constructing reliable, query-specific rubrics remains a bottleneck because existing methods often rely on external annotations or model training, creating a rubric-quality gap where plausible rubrics may fail to distinguish quality or reward optional styles. This paper introduces RUBRICS ON TRIAL, a query-only framework that evolves a rubric set from an empty set by deriving supervision solely from synthetic rubric-conditioned response pairs and validating each proposed rubric before adding it, thereby screening out non-discriminative or style-only candidates.

The gist: RUBRICS ON TRIAL is a query-only framework that evolves a rubric set from an empty set without external annotations or model training.

How it works

The framework operates as a role-specialized multi-agent system with three LLM roles: a rubric proposer, response generators, and a rubric-blind pairwise judge. A deterministic controller maintains the evolution tree structure, denoted as Tt, the active leaf set Rt, and the evolution memory Ht. The core decision at each step is whether a candidate rubric should be added or if an existing bundled rubric should be split into atomic children.

1. The proposer emits a patch pt, which is either an ADD(r) patch proposing one new atomic rubric r, or a SPLIT(rp → m) patch proposing retiring a bundled parent and replacing it with m atomic children.

2. For any candidate rubric r, the controller constructs two complementary pairs of responses conditioned on the trial background Bt:

a. The local pair consists of response a+ (a response from scratch that maximizes quality while passing Bt and r) and a− (the smallest coherent edit to a+ that violates r while preserving its strategy and satisfying Bt). This tests whether violating r makes an otherwise unchanged response worse.

b. The alternative-answer pair consists of response b− (the strongest response from scratch that passes Bt while failing r) and b+ (a minimal repair of b− to pass r while preserving its strategy, content, and satisfaction of Bt). This tests whether r is over-specific or optional or style-only.

How it works (Continued)

The rubric-blind judge evaluates these pairs independently. For each pair, the judge receives only q and the two responses; it does not see r, Bt, construction instructions, or plus/minus labels. A fixed lookup rule determines acceptance based on pairwise outcomes:

(a+ ≻ a− b+ ≻ b−)

If both pairs prefer the response satisfying r (i.e., satisfying r improves response quality), the candidate rubric is ACCEPTED.

(a+ ≃ a− b+ ≃ b−)

If both pairs remain equally good with or without r, the rubric is REJECTED because it is deemed optional or style-only.

(Either pair prefers its minus response)

If either pair prefers the response violating r, the rubric is REJECTED because it makes a response worse and is thus harmful.

How it works (Continued)

Accepted ADD operations attach a new rubric as an active leaf in Tt+1. Rejected proposals leave Tt unchanged, and every decision and its reason enter evolution memory Ht+1. The evolution process continues until the budget is exhausted, yielding the final active-leaf set Rt.

How it works (Continued)

The framework employs a tree-structured evolution process where accepted ADD operations update the active leaves, and SPLIT operations commit children only when they pass. A harmful or inconclusive child result leaves the entire patch unchanged to guide subsequent proposals. The evolution memory Ht+1 records the patch, each decision, its plain-language reason, and a compact counterexample.

Empirical Effectiveness

Experiments across five preference benchmark suites and seven evaluation sets demonstrate that RUBRICS ON TRIAL achieves the best average accuracy and leads on six of seven evaluation sets. Specifically, it improves over the strongest baseline in each of these columns by 1.31–4.19 percentage points compared to existing methods. RUBRICS ON TRIAL outperforms every trained open-weight generator on all seven sets, with the only exception being JudgeBench, where it ranks second to TICK. This indicates that pairwise validation enables RUBRICS ON TRIAL to produce consistently high-quality rubrics.

Limitations & Future Work

The current experiments focus primarily on preference benchmarks, which measure whether generated rubrics can discriminate between responses in agreement with human preferences. Future work will evaluate RUBRICS ON TRIAL in rubric-based post-training. The researchers also plan to conduct systematic component ablations or sensitivity analyses to isolate the contributions of the local pair, alternative-answer pair, tree-structured evolution, and evolution memory. Finally, broader model families and task domains are needed to characterize its robustness.

REFERENCES

Jonathan Cook, Tim Rocktaschel, Jakob Foerster, Dennis Aumiller, and Alex Wang. TICKing ¨ all the boxes: Generated checklists improve LLM evaluation and generation. arXiv preprint arXiv:2410.03608, 2024.

Dazhi Fu, Jiuding Yang, Yiwen Guo, and Jicong Fan. Many voices, one reward: Multi-role rubric generation for LLM judging and reward modeling. arXiv preprint arXiv:2607.01830, 2026.

Shanghua Gao, Yuchang Su, Pengwei Sui, Curtis Ginder, and Marinka Zitnik. Qworld: Questionspecific evaluation criteria for LLMs. arXiv preprint arXiv:2603.23522, 2026.

Anisha Gunjal, Anthony Wang, Elaine Lau, Vaskar Nath, Yunzhong He, Bing Liu, and Sean M. Hendryx. Rubrics as rewards: Reinforcement learning beyond verifiable domains. In The Fourteenth International Conference on Learning Representations, 2026.

Haoxiang Jiang, Zihan Dong, Tianci Liu, Wanying Wang, Ran Xu, Tony Yu, Linjun Zhang, and Haoyu Wang. RUBRIC-ARROW: Alternating pointwise rubric reward modeling for LLM post-training in nonverifiable domains. arXiv preprint arXiv:2605.29156, 2026.

Nathan Lambert, Valentina Pyatkin, Jacob Morrison, LJ Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, Noah A. Smith, and Hannaneh Hajishirzi. RewardBench: Evaluating reward models for language modeling. arXiv preprint arXiv:2403.13787, 2024.

Shuyue Stella Li, Rui Xin, Teng Xiao, Yike Wang, Rulin Shao, Zoey Hao, Melanie Sclar, Sewoong Oh, Faeze Brahman, Pang Wei Koh, and Yulia Tsvetkov. EvoLM: Self-evolving language models through co-evolved discriminative rubrics. arXiv preprint arXiv:2605.03871, 2026a.

Xiaoxi Li, Jiajie Jin, Guanting Dong, Hongjin Qian, Yongkang Wu, Ji-Rong Wen, Yutao Zhu, and Zhicheng Dou. WebThinker: Empowering large reasoning models with deep research capability. In Advances in Neural Information Processing Systems, volume 38, 2025.

Xiaoxi Li, Wenxiang Jiao, Jiarui Jin, Guanting Dong, Jiajie Jin, Yinuo Wang, Hao Wang, Yutao Zhu, Ji-Rong Wen, Yuan Lu, and Zhicheng Dou. DeepAgent: A general reasoning agent with scalable toolsets. In Proceedings of the ACM Web Conference 2026

Dengcan Liu, Fengkai Yang, Xiaohan Wang, Shurui Yan, Jiajun Chai, Jiahao Li, Yikun Ban, Zhendong Mao, Wei Lin, and Guojun Yin. CDRRM: Contrast-driven rubric generation for reliable and interpretable reward modeling. arXiv preprint arXiv:2603.08035

Tianci Liu, Ran Xu, Tony Yu, Ilgee Hong, Carl Yang, Tuo Zhao, and Haoyu Wang. OpenRubrics: Towards scalable synthetic rubric generation for reward modeling and LLM alignment. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 17417–17437. Association for Computational Linguistics, 2026b doi: 10.18653/v1/2026.acl-long.791.

Yantao Liu, Zijun Yao, Rui Min, Yixin Cao, Lei Hou, and Juanzi Li. RM-Bench: Benchmarking reward models of language models with subtlety and style. arXiv preprint arXiv:2410.

1. No external supervision is required for rubric generation, allowing systems to evolve a rubric set from an empty set without external annotations or model training. This enables query-only rubric evolution, which addresses the bottleneck of needing costly human-written rubrics.

2. A complementary two-pair gate validates each candidate rubric before it enters the active set by constructing pairs where a local pair begins with a strong response a + satisfying the background rubrics together with r, then makes the smallest coherent edit a − that violates r and an alternative pair is instead written independently from scratch to test if r is over-specific.

3. The system utilizes a tree-structured evolution process where accepted ADD and SPLIT operations update the set, ensuring that rejected proposals provide feedback for subsequent proposals, allowing the framework to guide future proposals without entering the active rubric set.