Few-class Fidelity: Evaluating Explanations of Real-conditions CNN classifiers with Optimized Perturbations
cs.CV, cs.AI
Submitted: 2026-06-23
Updated: 2026-09-21
Comments: Under consideration at Pattern Recognition Letters
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: The wide use of Convolutional Neural Networks (CNN) in numerous domains and real-world classification applications is justified by their high precision and automation speed, helping users concentrate
Terminology
Abstract
The wide use of Convolutional Neural Networks (CNN) in numerous domains and real-world classification applications is justified by their high precision and automation speed, helping users concentrate on higher-expertise tasks. To better understand the models and avoid bias during deployment, eXplainable Artificial Intelligence (XAI) techniques can be used after training. But as the list of XAI solutions expand, comparisons between them diverge, and consensus over their evaluation cannot be reached. This paper proposes a variation of Fidelity-based XAI metrics, with a focus on real-conditions applications, where the number of classes is often low. The approach generates in-distribution, uncertainty-provoking perturbations, to ensure proper measurement of the XAI methods faithfulness. As demonstration of the evaluation framework usefulness, it is compared with human-centric object localization and segmentation metrics. Once applied to both medical and natural imaging applications, it highlights the intricate correlation between domain, data curation, and XAI solution choices in order to validate training of a new CNN model.
Sources
- Towards better understanding of gradient-based attribution methods for Deep Neural Networks
- Explaining Image Classifiers by Counterfactual Generation
- A Benchmark for Interpretability Methods in Deep Neural Networks
- LNDb: A Lung Nodule Database on Computed Tomography
- RISE: Randomized Input Sampling for Explanation of Black-box Models
- Resisting Out-of-Distribution Data Problem in Perturbation of XAI
- IROF: a low resource evaluation metric for explanation methods
- Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
- SmoothGrad: removing noise by adding noise
- Visualizing and Understanding Convolutional Networks
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models