Performance uncertainty in medical image analysis: a large-scale investigation of confidence intervals

arXiv:2601.17103 · cs.CV, cs.LG · Submitted 2026-01-23 · Read on arXiv

cs.CV, cs.LG

Submitted: 2026-01-23

Updated: 2026-08-26

Code: https://github.com/Project-MONAI/MetricsReloaded

Terminology

• Accuracy — 92× in this paper · explained in 1 episode
• AUC — 53× in this paper · explained in 2 episodes
• Benchmark — 38× in this paper · explained in 2 episodes
• F1 Score — 33× in this paper · explained in 3 episodes
• Aggregation — 24× in this paper · explained in 1 episode
• Empirical Coverage — 14× in this paper · explained in 2 episodes
• Uncertainty Quantification — 9× in this paper · explained in 8 episodes
• Artificial Intelligence — 8× in this paper
• Core — 8× in this paper · explained in 2 episodes
• Average Symmetric Surface Distance — 3× in this paper
• Benchmarking — 3× in this paper · explained in 3 episodes
• Conformal Prediction — 3× in this paper · explained in 5 episodes
• Matthews Correlation Coefficient — 3× in this paper
• Normalized Surface Distance — 3× in this paper
• Artificial Intelligence (AI) — 2× in this paper · explained in 2 episodes
• Average Precision — 2× in this paper
• Consistency — 2× in this paper · explained in 3 episodes
• Dice Similarity Coefficient — 2× in this paper
• Fingerprinting — 2× in this paper · explained in 2 episodes
• Generalization — 2× in this paper · explained in 13 episodes
• Hausdorff Distance — 2× in this paper
• Helmholtz Imaging — 2× in this paper
• Mean Average Surface Distance — 2× in this paper
• Failure Modes — 1× in this paper · explained in 3 episodes
• Fisher Information — 1× in this paper · explained in 3 episodes
• Food and Drug Administration — 1× in this paper
• Generative AI — 1× in this paper · explained in 2 episodes
• Large Language Models (LLMs) — 1× in this paper · explained in 6 episodes
• National Center for Tumor Diseases — 1× in this paper
• Robustness — 1× in this paper · explained in 5 episodes
• Score Function — 1× in this paper · explained in 4 episodes
• Verification — 1× in this paper · explained in 2 episodes

Sources

• Introduction to Concentration Inequalities
• MONAI: An open-source framework for deep learning in healthcare
• False Promises in Medical Imaging AI? Assessing Validity of Outperformance Claims
• Biomedical image analysis competitions: The state of current participation practice
• Beyond Knowledge Silos: Task Fingerprinting for Democratization of Medical Imaging AI
• Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations
• Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning
• Constructing Confidence Intervals for 'the' Generalization Error -- a Comprehensive Benchmark Study
• COVID-CT-Dataset: A CT Scan Dataset about COVID-19

Related papers

• Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
• AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
• Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
• MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
• TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
• A Survey on Efficient Vision-Language-Action Models