Generalized Correctness Models: Learning Calibrated and Model-Agnostic Correctness Predictors from Historical Patterns
cs.CL, cs.AI
Submitted: 2025-09-29
Updated: 2026-09-12
Comments: ICML 2026. Code: https://github.com/The-Inscrutable-X/CalibratedModelAgnosticCorrectness
Code: https://github.com/The-Inscrutable-X/CalibratedModelAgnosticCorrectness
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Terminology
Sources
- The Internal State of an LLM Knows When It's Lying
- InternalInspector $I^2$: Robust Confidence Estimation in LLMs through Internal States
- Skill-Based Mixture-of-Experts: Adaptive Routing for Heterogeneous Reasoning via Inferred Skills
- Learning to Route LLMs with Confidence Tokens
- Training Verifiers to Solve Math Word Problems
- Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty
- GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
- On Calibration of Modern Neural Networks
- Deep Anomaly Detection with Outlier Exposure
- Measuring Massive Multitask Language Understanding
- LoRA: Low-Rank Adaptation of Large Language Models
- RouterBench: A Benchmark for Multi-LLM Routing System
- TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
- Language Models (Mostly) Know What They Know
- Large Language Models Must Be Taught to Know What They Don't Know
- Inference-Time Intervention: Eliciting Truthful Answers from a Language Model
- Confidence Is All You Need: Few-Shot RL Fine-Tuning of Language Models
- ConfTuner: Training Large Language Models to Express Their Confidence Verbally
- AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
- Uncertainty Estimation and Quantification for LLMs: A Simple Supervised Approach
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering