Confidence Calibration of Deep Learning Systems

This thesis explores novel methods for improving confidence calibration under challenging conditions such as label noise, domain shifts, and privacy constraints. The work addresses the critical need for reliable model confidence in high-stakes applications like medical imaging, where a model's predicted probabilities must accurately reflect its likelihood of correctness.

The thesis first addresses confidence calibration in the presence of label noise. The authors propose a calibration framework that accounts for label noise by leveraging an estimated noise model. They demonstrate how to reconstruct noise-free confidence estimates by modeling the relationship between noisy and clean label distributions. This idea is extended to Conformal Prediction (CP), a framework that provides set-valued predictions with a guaranteed level of coverage. A noise-aware conformal prediction approach is introduced that estimates the true conformity scores despite label noise, allowing for efficient and reliable uncertainty quantification.

Next, the research investigates confidence calibration in unsupervised domain adaptation (UDA), where a model trained on a labeled source domain is adapted to an unlabeled target domain. Since traditional calibration methods require labeled validation data from the target domain, which is unavailable, the authors develop an approach that estimates the target domain accuracy based on the model’s performance in the source domain and known domain discrepancies. This allows for direct calibration of model confidence without access to target domain labels.

The study is further extended to privacy-preserving settings, where individual user labels and model outputs must be protected. A locally differentially private conformal prediction framework is proposed that ensures valid uncertainty quantification while maintaining rigorous privacy guarantees. This approach balances the trade-offs between privacy, computational feasibility, and prediction reliability, making it applicable to sensitive medical data applications.

Through extensive experiments on natural and medical imaging datasets, the proposed methods are shown to significantly improve calibration robustness under both label noise and domain shift conditions. The thesis provides theoretical guarantees and empirical validations that bridge the gap between theoretical calibration guarantees and practical deployment in safety-critical environments. The findings contribute to the development of reliable, privacy-preserving, and noise-resilient calibration frameworks, enhancing the trustworthiness of neural network predictions in real-world medical and high-stakes applications.

Improvements to AI Systems:

1. Noise-Aware Confidence Calibration Module: Integrate a trainable noise-model estimator into the AI’s output layer. This module reconstructs clean-label probabilities from noisy training labels, enabling the AI to output calibrated confidence scores even when trained on corrupted datasets (e.g., mislabeled medical records or web-scraped images).

2. Domain-Shift Calibrated Predictions: Add a domain-discrepancy estimator that uses source-domain accuracy and feature-space distance metrics to recalibrate confidence on unlabeled target domains. The AI can then self-adjust its confidence without requiring any labeled target data, making it reliable when deployed in new hospitals, sensors, or imaging protocols.

3. Privacy-Preserving Uncertainty Quantification: Implement a locally differentially private conformal prediction layer. This layer adds calibrated noise to conformity scores while maintaining a rigorous coverage guarantee, allowing the AI to output prediction sets (e.g., “top-3 possible diagnoses”) with provable validity even when user data must remain private.

4. Label-Noise-Robust Conformal Prediction: Replace standard conformal score computation with a noise-corrected version that estimates true conformity scores from noisy labels. This yields prediction sets with exact coverage guarantees despite label corruption, reducing over-confidence in high-stakes decisions.

Capabilities of the Improved AI System:

• In medical imaging, the AI can flag its own low-confidence predictions (e.g., “I am only 60% sure this is a malignant lesion”) even when training labels were noisy or when deployed on a new MRI machine, without needing any labeled examples from that machine.

• The AI can provide valid, set-valued predictions (e.g., “This could be pneumonia or bronchitis”) with a user-specified confidence level (e.g., 90% coverage) while guaranteeing that individual patient data cannot be reverse-engineered from the model’s outputs.

• The system remains trustworthy under distribution shift: if deployed on a different demographic or imaging protocol, it automatically recalibrates its confidence using only unlabeled data, preventing silent overconfidence.

• In federated or decentralized settings, the AI can produce uncertainty estimates that are mathematically valid under strict differential privacy budgets, enabling safe deployment in clinical trials or multi-site studies where data cannot leave the institution.