JEV vs. LLMs as Rubric Judges: Cheaper, Faster, and Wrong in the Same Places

arXiv:2609.29769 · cs.CL · Submitted 2026-09-24 · Read on arXiv

cs.CL

Submitted: 2026-09-24

Updated: 2026-09-28

Terminology

Accuracy — 128× in this paper · explained in 1 episode
LLM judge — 117× in this paper · explained in 2 episodes
Abstention — 21× in this paper · explained in 1 episode
Benchmark — 18× in this paper · explained in 2 episodes
Calibration — 8× in this paper · explained in 7 episodes
Provenance — 7× in this paper · explained in 3 episodes
Ablation — 3× in this paper · explained in 1 episode
Deferral — 3× in this paper · explained in 2 episodes
LLM-as-a-Judge — 3× in this paper · explained in 3 episodes
Aggregation — 2× in this paper · explained in 1 episode
Alignment — 2× in this paper · explained in 1 episode
Generative AI — 2× in this paper · explained in 2 episodes
Reinforcement Learning — 2× in this paper · explained in 4 episodes
Clustering — 1× in this paper · explained in 2 episodes
Decode — 1× in this paper · explained in 2 episodes
Failure Modes — 1× in this paper · explained in 3 episodes
Intelligence Advanced Research Projects Activity — 1× in this paper
Tokenization — 1× in this paper · explained in 2 episodes
Trace — 1× in this paper · explained in 2 episodes
Transformer — 1× in this paper · explained in 2 episodes
Verification — 1× in this paper · explained in 2 episodes

Sources

Related papers