Are explainable AI (XAI) evaluation strategies aligned? Comparing subjective, objective, and mathematical evaluation measures using saliency maps

arXiv:2504.17023 · cs.HC, cs.AI · Submitted 2025-04-23 · Read on arXiv

cs.HC, cs.AI

Submitted: 2025-04-23

Updated: 2026-09-14

Comments: 31 pages, 8 figures, 6 tables

Journal ref: Kares, Felix, et al. "Are explainable AI (XAI) evaluation strategies aligned? Comparing subjective, objective, and mathematical evaluation measures using saliency maps." Computers in Human Behavior (2026): 109163

Code: https://github.com/jacobgil/pytorch-grad-cam

License: http://creativecommons.org/licenses/by-nc-nd/4.0/

The gist: The evaluation of explainable AI (XAI) approaches often relies on three families of methods: subjective measures (e.g., questionnaires on trust or satisfaction), objective measures (e.g., task

Terminology

Accuracy — 49× in this paper · explained in 1 episode
Fidelity — 15× in this paper · explained in 2 episodes
Robustness — 12× in this paper · explained in 5 episodes
Explainability — 8× in this paper · explained in 3 episodes
International Conference on Neural Information Processing Systems — 7× in this paper
Interpretability — 6× in this paper · explained in 15 episodes
Benchmarking — 5× in this paper · explained in 3 episodes
Conference on Computer Vision and Pattern Recognition — 5× in this paper
Explainable AI (XAI) — 5× in this paper · explained in 7 episodes
Alignment — 4× in this paper · explained in 1 episode
Adam — 3× in this paper · explained in 2 episodes
Conference on Human Factors in Computing Systems — 3× in this paper
Accountability — 2× in this paper · explained in 2 episodes
Adversarial Robustness — 2× in this paper · explained in 3 episodes
Energy Pointing Game — 2× in this paper
European Conference on Computer Vision — 2× in this paper
IEEE International Requirements Engineering Conference — 2× in this paper
Local Interpretable Model-Agnostic Explanations — 2× in this paper
Model-Agnostic — 2× in this paper · explained in 2 episodes
Benchmark — 1× in this paper · explained in 2 episodes
Consistency — 1× in this paper · explained in 3 episodes
Convolutional Neural Networks (CNNs) — 1× in this paper · explained in 2 episodes
EU General Data Protection Regulation — 1× in this paper
Gradient-weighted Class Activation Mapping — 1× in this paper
IEEE International Conference on Computer Vision — 1× in this paper
Information and Communication Technology — 1× in this paper
International Joint Conference on Artificial Intelligence — 1× in this paper
Meeting of the Association for Computational Linguistics — 1× in this paper
Nadia — 1× in this paper · explained in 2 episodes
Sparsity — 1× in this paper · explained in 4 episodes

Abstract

The evaluation of explainable AI (XAI) approaches often relies on three families of methods: subjective measures (e.g., questionnaires on trust or satisfaction), objective measures (e.g., task performance metrics), and mathematical metrics (e.g., for faithfulness). Yet, it remains unclear how these families align or diverge in practice. In a preregistered between-subjects study (N=166), we use three established saliency map techniques (LIME, Grad-CAM, Guided Backpropagation) as a testbed to examine this issue. We find that each family of methods leads to different conclusions: participants reported no differences in trust or satisfaction, Grad-CAM improved user performance, while mathematical metrics favored Guided Backpropagation. At the same time, mathematical metrics were only partially related to user performance, and these relationships were sometimes counterintuitive. Our findings highlight the methodological importance of comparing subjective, objective, and mathematical approaches when evaluating XAI, illustrating both tensions and aspects that are aligned. We discuss implications for XAI evaluation frameworks.

Are explainable AI (XAI) evaluation strategies aligned? Comparing subjective, objective, and mathematical evaluation measures using saliency maps

Terminology

Abstract

Sources

Related papers