Training LLM Judges from Language Feedback via Position-Selective Self-Distillation
cs.CL, cs.LG
Submitted: 2026-09-30
Updated: 2026-09-30
Terminology
Sources
- Constitutional AI: Harmlessness from AI Feedback
- RM-R1: Reward Modeling as Reasoning
- The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- Reward Reasoning Model
- Where Hindsight Credit Can Reside: A Signed-Capacity View of Token Updates in RLVR
- Think-RM: Enabling Long-Horizon Reasoning in Generative Reward Models
- Reinforcement Learning via Self-Distillation
- Asymmetric On-Policy Distillation: Bridging Exploitation and Imitation at the Token Level
- Entropy-Aware On-Policy Distillation of Language Models
- Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs?
- Efficient Memory Management for Large Language Model Serving with PagedAttention
- Feedback Descent: Open-Ended Text Optimization via Pairwise Comparison
- From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipeline
- TIS-DPO: Token-level Importance Sampling for Direct Preference Optimization With Estimated Weights
- OpenRubrics: Towards Scalable Synthetic Rubric Generation for Reward Modeling and LLM Alignment
- When Are Teacher Tokens Reliable? Position-Weighted On-Policy Self-Distillation for Reasoning
- RM-Bench: Benchmarking Reward Models of Language Models with Subtlety and Style
- Understanding R1-Zero-Like Training: A Critical Perspective
- Decoupled Weight Decay Regularization
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering