From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI
От анекдотических свидетельств к количественным методам оценки: систематический обзор оценки объяснимого искусственного интеллекта
2023-02-24
SCID: 54.1/agrwekjk
Discuss with AI
Co-12 propertiesXAI evaluationexplainable artificial intelligencequantitative evaluation methodssystematic review
Figures from the paper
Abstract (AI)
The rising popularity of explainable artificial intelligence (XAI) to understand high-performing black boxes raised the question of how to evaluate explanations of machine learning (ML) models. While interpretability and explainability are often presented as a subjectively validated binary property, we consider it a multi-faceted concept. We identify 12 conceptual properties, such as Compactness and Correctness, that should be evaluated for comprehensively assessing the quality of an explanation. Our so-called Co-12 properties serve as categorization scheme for systematically reviewing the evaluation practices of more than 300 papers published in the past 7 years at major AI and ML conferences that introduce an XAI method. We find that one in three papers evaluate exclusively with anecdotal evidence, and one in five papers evaluate with users. This survey also contributes to the call for objective, quantifiable evaluation methods by presenting an extensive overview of quantitative XAI evaluation methods. Our systematic collection of evaluation methods provides researchers and practitioners with concrete tools to thoroughly validate, benchmark, and compare new and existing XAI methods. The Co-12 categorization scheme and our identified evaluation methods open up opportunities to include quantitative metrics as optimization criteria during model training to optimize for accuracy and interpretability simultaneously.
Key Findings
1
One-third of reviewed papers rely exclusively on anecdotal evidence, while only one-fifth evaluate explanations with users.
2
The Co-12 framework categorizes evaluation practices across more than 300 XAI papers published at major AI and machine learning conferences over seven years.
3
The Co-12 scheme and quantitative metrics could enable explanation properties to become optimization criteria for jointly improving model accuracy and interpretability.
4
The review conceptualizes explanation quality as multifaceted and identifies 12 properties, including Compactness and Correctness, for comprehensive XAI evaluation.
5
The review systematically compiles quantitative XAI evaluation methods that can support validation, benchmarking, and comparison of explanation techniques.
Research Object
evaluation practices and methods for explanations of machine learning models in explainable artificial intelligence (XAI)
Research Subject
the multifaceted quality of XAI explanations, including their quantitative evaluation across properties such as Compactness and Correctness
Publication Details
Publication Date
2023-02-24
Journal
Publisher
ISSN
Cited by
590
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai9
A Unified Approach to Interpreting Model Predictions2017
Greedy function approximation: A gradient boosting machine.2001
"Why Should I Trust You?": Explaining the Predictions of Any Classifier2016
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)2018
Machine Learning Interpretability: A Survey on Methods and Metrics2019
Explainable Machine Learning for Scientific Insights and Discoveries2020
A Survey on the Explainability of Supervised Machine Learning2021
A Survey of Methods for Explaining Black Box Models2018
Evaluating the Quality of Machine Learning Explanations: A Survey on Methods and Metrics2021
Cited by6
Explainable Artificial Intelligence (XAI): What we know and what is left to attain Trustworthy Artificial Intelligence2023
FUTURE-AI: international consensus guideline for trustworthy and deployable artificial intelligence in healthcare2025
Explainable Artificial Intelligence (XAI) 2.0: A manifesto of open challenges and interdisciplinary research directions2024
Recent Applications of Explainable AI (XAI): A Systematic Literature Review2024
Explainable artificial intelligence techniques for interpretation of food models: a review2026
Explainable Artificial Intelligence (XAI) for EEG Analysis: A Survey on Recent Trends and Advancements2026