Mitigating False Credit Propagation: Probabilistic Graphical Reward Aggregation for Rubric-Based Reinforcement Learning

arXiv:2606.03361 · cs.LG · Submitted 2026-06-02 · Read on arXiv

cs.LG

Submitted: 2026-06-02

Updated: 2026-08-26

Code: https://github.com/LvCan926/GEAR

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Terminology

• Aggregation — 135× in this paper · explained in 1 episode
• Benchmark — 43× in this paper · explained in 2 episodes
• Graph Construction — 15× in this paper · explained in 3 episodes
• Core — 11× in this paper · explained in 2 episodes
• False Credit Propagation — 11× in this paper
• Reinforcement Learning — 11× in this paper · explained in 4 episodes
• Ablation — 8× in this paper · explained in 1 episode
• GRPO — 6× in this paper · explained in 6 episodes
• Grounding — 4× in this paper · explained in 3 episodes
• LLM-as-a-Judge — 4× in this paper · explained in 3 episodes
• Calibration — 3× in this paper · explained in 7 episodes
• Ablation Studies — 2× in this paper · explained in 1 episode
• Alignment — 1× in this paper · explained in 1 episode
• Benchmarking — 1× in this paper · explained in 3 episodes
• Credit assignment — 1× in this paper · explained in 3 episodes
• Fine-Tuning — 1× in this paper · explained in 5 episodes
• Generalization — 1× in this paper · explained in 13 episodes
• Generative Model — 1× in this paper · explained in 2 episodes
• LLM judge — 1× in this paper · explained in 2 episodes
• Sparsity — 1× in this paper · explained in 4 episodes
• Topological Order — 1× in this paper · explained in 2 episodes
• alpha — 1× in this paper · explained in 1 episode

Sources

• HealthBench: Evaluating Large Language Models Towards Improved Human Health
• Constitutional AI: Harmlessness from AI Feedback
• From Rubrics to Reliable Scores: Evidence-Grounded Text Evaluation with LLM Judges
• Reinforcement Learning with Rubric Anchors
• Multidimensional Rubric-oriented Reward Model Learning via Geometric Projection Reference Constraints
• Prometheus: Inducing Fine-grained Evaluation Capability in Language Models
• Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models
• The Llama 3 Herd of Models
• A Survey on LLM-as-a-Judge
• RewardBench: Evaluating Reward Models for Language Modeling
• Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains
• RubricHub: A Comprehensive and Highly Discriminative Rubric Dataset via Automated Coarse-to-Fine Generation
• DEER: A Benchmark for Evaluating Deep Research Agents on Expert Report Generation
• G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment
• Training language models to follow instructions with human feedback
• AdvancedIF: Rubric-Based Benchmarking and Reinforcement Learning for Advancing LLM Instruction Following
• RubricEval: A Rubric-Level Meta-Evaluation Benchmark for LLM Judges in Instruction Following
• Qwen3 Technical Report
• Qwen2.5 Technical Report
• A Simple "Motivation" Can Enhance Reinforcement Finetuning of Large Reasoning Models

Related papers

• Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
• AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
• Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
• Convergence issues in Relational Concept Analysis based on AOC-posets
• Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
• Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks