Multi-Expert Conformal Risk Control for Pairwise LLM Judging in Open-Ended Dialogue
cs.CL
Submitted: 2026-08-27
Updated: 2026-08-27
Terminology
Sources
- SCOPE: Selective Conformal Optimized Pairwise LLM Judging
- Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
- Deep Think with Confidence
- The Llama 3 Herd of Models
- HEART: A Unified Benchmark for Assessing Humans and LLMs in Emotional Support Dialogue
- Gemma 3 Technical Report
- Scalable Best-of-N Selection for Large Language Models via Self-Certainty
- Conformal Prediction with Large Language Models for Multi-Choice Question Answering
- LLM Evaluators Recognize and Favor Their Own Generations
- Who can we trust? LLM-as-a-jury for Comparative Assessment
- Conformal Language Modeling
- EmPO: Emotion Grounding for Empathetic Response Generation through Preference Optimization
- Large Language Models are Inconsistent and Biased Evaluators
- Self-Consistency Improves Chain of Thought Reasoning in Language Models
- Qwen2.5 Technical Report
- Auto-Arena: Automating LLM Evaluations with Agent Peer Battles and Committee Discussions
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering