Binary Agreement for Efficient Unsupervised Confidence Calibration of Reasoning LLMs
summary
The gist
The gist The method introduces an unsupervised confidence calibration technique for reasoning LLMs that derives a self-consistency-based proxy target from offline sampling on unlabeled data to
In short
The method introduces Binary Agreement for Self-consistency Calibration (BASC), an unsupervised technique to calibrate confidence for reasoning LLMs without needing labeled data or expensive test-time sampling. It uses repeated sampling on unlabeled data to create a lightweight predictor that estimates self-consistency as a proxy for correctness, achieving strong calibration across multiple models and tasks.
Key concepts
- Self-consistency score s(x, a)
- This score measures how often an answer 'a' is selected when the same input 'x' is sampled multiple times. It quantifies the internal agreement among different responses for a given query. The paper uses this offline to build a prediction model that approximates this agreement.
- Weakly labeled examples
- The method transforms unlabeled data into weakly labeled examples by selecting an input, a response, and its calculated self-consistency score from the offline sampling process. These synthetic labels allow the system to train a predictor without needing ground-truth correctness labels.
- Binary Isotonic Recalibration
- This is a distribution-free calibration guarantee used at deployment. It ensures that the confidence scores generated by the model are reliably mapped to actual correctness, even if the underlying self-consistency signal has some noise or distribution shift.
Terminology used across episodes
This episode discusses
- Binary Agreement for Efficient Unsupervised Confidence Calibration of Reasoning LLMs · Paper Radio
- Conformal Prediction for Natural Language Processing: A Survey
- Large language model validity via enhanced conformal prediction methods
- Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty
- Statistical Methods in Generative AI
- Domain-Adversarial Training of Neural Networks
- An Overview of Large Language Models for Statisticians
- AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions
- Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMs
- Verified Uncertainty Calibration
- Taming Overconfidence in LLMs: Reward Calibration in RLHF
- Deep Unsupervised Domain Adaptation: A Review of Recent Advances and Perspectives
- Your Pre-trained LLM is Secretly an Unsupervised Confidence Calibrator
- Reasoning about Uncertainty: Do Reasoning Models Know When They Don't Know?
- Revisiting the Calibration of Modern Neural Networks
- AIMO-2 Winning Solution: Building State-of-the-Art Mathematical Reasoning Models with OpenMathReasoning dataset
- NVIDIA Nemotron 3: Efficient and Open Intelligence
- BaseCal: Unsupervised Confidence Calibration via Base Model Signals
- Self-Consistency Improves Chain of Thought Reasoning in Language Models
- PolyMath: Evaluating Mathematical Reasoning in Multilingual Contexts
- Crowdsourcing Multiple Choice Science Questions
The paper
Binary Agreement for Efficient Unsupervised Confidence Calibration of Reasoning LLMs · Read on arXiv
Thomas Zollo, Jimmy Wang, Richard Zemel
Columbia University
Reasoning language models often lack the calibrated confidence needed for reliable deployment, particularly when correctness labels are unavailable and sampling budgets are limited. We introduce Binary Agreement for Self-consistency Calibration (BASC), which uses self-consistency as an unlabeled proxy for correctness without constructing costly self-consistency targets for each calibration query. Our key observation is that agreement between an anchor response and a single independent probe is a Bernoulli observation whose conditional mean is the anchor's latent self-consistency. Pooling these observations across unlabeled queries, BASC supports both recalibrating existing confidence scores and learning more discriminative scores before recalibration, using as few as two responses per calibration query and only one at deployment. The use of binary isotonic recalibration provides a distribution-free calibration guarantee with respect to latent self-consistency. Across 5 tasks and 9 reasoning models, BASC substantially improves calibration with respect to correctness.
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "Binary Agreement for Efficient Unsupervised Confidence Calibration of Reasoning LLMs".
Jane: Here's the gist of the paper. The gist The method introduces an unsupervised confidence calibration technique for reasoning LLMs that derives a self-consistency-based proxy target from offline sampling on unlabeled data to distill it into a lightweight deployment-time predictor, which substantially outperforms baselines in various settings.
Tom: How it works 1.
Jane: The core idea treats repeated sampling not as a deployment-time inference procedure but as an offline source of unlabeled supervision, where the approach uses "repeated sampling on unlabeled examples to estimate a self-consistency signal".
Tom: 2.
Paper summary: Tom: We've covered the high-level idea and the initial wins with self-consistency. Now let's dig into exactly how they build that predictor from the offline sampling >
Jane: The paper, "Binary Agreement for Efficient Unsupervised Confidence Calibration of Reasoning LLMs," focuses on treating repeated sampling not as a test-time inference procedure but as an offline source of unlabeled supervision >
Lu: They formally define a self-consistency score for an input answer pair as s(x, a) = one/k times the number of times answer a appears out of k samples where the answer is equal to the anchor answer > <ref:2604.19444#pg1>
Tom: Instead of computing this score at test time, they use it offline on unlabeled data to construct supervision for a single-pass predictor by selecting a sampled response i = (i, i) together with its score s i = s(x i, i) >
Jane: This process yields weakly labeled examples of the form (x i, i, s i), and then they train a predictor f phi of (x, r, a) to estimate the empirical support for the response's answer under repeated sampling >
Meng: So it’s not training on perfect labels; it’s training on these weakly labeled examples derived from the self-consistency signal themselves >
Lu: This is how they distill that signal into a deployment-time confidence predictor f phi which maps the input and a response to an estimated confidence value >
Tom: The key is that this predictor, trained offline, replaces the need for expensive test-time sampling or generating many samples per query >
Jane: So at deployment time, you give it a new input x and a response (r, a), and the confidence score is reported as c(x, y) = f phi(x, r, a) >
Meng: It sounds like they’re trading the cost of runtime sampling for the cost of offline training on unlabeled data >
Tom: And empirically across five math and question-answering tasks using nine models from the Qwen, DeepSeek, and Nemotron families, they show that self-consistency is generally well-calibrated with respect to correctness >
Jane: And they found that this signal can be distilled into a single-generation confidence predictor that substantially outperforms relevant unsupervised baselines, including under distribution shift and in blackbox settings >
Lu: That's the second main result they highlight, showing the power of distilling that self-consistency signal >
Conclusion: Tom: So to wrap up this paper on "Binary Agreement for Efficient Unsupervised Confidence Calibration of Reasoning LLMs," it suggests that we don't necessarily need expensive labeled calibration data or compute-intensive reinforcement learning training to get reliable confidence scores from reasoning LLMs >
Jane: The authors show that by using offline sampling on unlabeled data to derive a self-consistency proxy target, you can distill that into a lightweight predictor for deployment >
Lu: They demonstrate that this method substantially outperforms unsupervised baselines, even when dealing with distribution shift and in blackbox settings >
Meng: For deployment folks, the implication is you can get better calibrated confidence estimates using only one generation at test time >
Tom: The whole point seems to be learning calibration from unlabeled data and deploying it with just one generation >
Jane: It’s a way of making the gap smaller to stronger supervised or multi-sample approaches while being much more practical for real applications, especially in selective prediction tasks >
More episodes
- 2507.11228-Gradient Descent on Logistic Regression: Do Large Step-Sizes Work with Data on the Sphere?
- 2507.21299-Blending data and physics for reduced-order modeling of systems with spatiotemporal chaotic dynamics
- 2509.00362-Optimized Weight Initialization on the Stiefel Manifold for Deep ReLU Neural Networks
- 2509.05865-The Measure of Deception: An Analysis of Data Forging in Machine Unlearning
- 2509.14203-Bellman Optimality of Average-Reward Robust Markov Decision Processes with a Constant Gain
- 2509.24608-Evaluating utility of clinical prediction models: A comparison of decision curve analysis and cost curves
- 2510.03811-Curriculum-Augmented GFlowNets For mRNA Sequence Generation
- 2510.10000-Wasserstein Robust Classification Error: Surrogate Loss Misalignment and Distributional Attacks
- 2510.13003-OPLoRA: Orthogonal Projection LoRA Prevents Catastrophic Forgetting during Parameter-Efficient Fine-Tuning
- 2511.14102-Specter: Fast MoE Inference via Quantization-Enabled Multi-Step Speculative Offloading