Evaluating the Quality of Machine Learning Explanations: A Survey on Methods and Metrics
Оценка качества объяснений в машинном обучении: обзор методов и метрик
2021-03-04
SCID: 54.1/22ru8u4r
Discuss with AI
attribution-based explanationsevaluation metricsexplainability evaluationmachine learning explanationsmodel-based explanations
Figures from the paper
Abstract (AI)
The most successful Machine Learning (ML) systems remain complex black boxes to end-users, and even experts are often unable to understand the rationale behind their decisions. The lack of transparency of such systems can have severe consequences or poor uses of limited valuable resources in medical diagnosis, financial decision-making, and in other high-stake domains. Therefore, the issue of ML explanation has experienced a surge in interest from the research community to application domains. While numerous explanation methods have been explored, there is a need for evaluations to quantify the quality of explanation methods to determine whether and to what extent the offered explainability achieves the defined objective, and compare available explanation methods and suggest the best explanation from the comparison for a specific task. This survey paper presents a comprehensive overview of methods proposed in the current literature for the evaluation of ML explanations. We identify properties of explainability from the review of definitions of explainability. The identified properties of explainability are used as objectives that evaluation metrics should achieve. The survey found that the quantitative metrics for both model-based and example-based explanations are primarily used to evaluate the parsimony/simplicity of interpretability, while the quantitative metrics for attribution-based explanations are primarily used to evaluate the soundness of fidelity of explainability. The survey also demonstrated that subjective measures, such as trust and confidence, have been embraced as the focal point for the human-centered evaluation of explainable systems. The paper concludes that the evaluation of ML explanations is a multidisciplinary research topic. It is also not possible to define an implementation of evaluation metrics, which can be applied to all explanation methods.
Key Findings
1
Human-centered evaluations increasingly focus on subjective measures such as user trust and confidence.
2
It identifies explainability properties from existing definitions and frames them as objectives that explanation-evaluation metrics should satisfy.
3
Quantitative evaluations of attribution-based explanations primarily assess explanation soundness through fidelity.
4
Quantitative evaluations of model-based and example-based explanations primarily measure interpretability parsimony or simplicity.
5
The survey concludes that explanation evaluation is multidisciplinary and no single metric implementation applies universally across explanation methods.
6
The survey provides a comprehensive overview of methods and metrics for evaluating the quality of machine-learning explanations.
Research Object
Machine learning explanations and explainable ML systems
Research Subject
Methods and metrics for evaluating explanation quality, including parsimony, fidelity, trust, and confidence
Publication Details
Publication Date
2021-03-04
Journal
Publisher
ISSN
Cited by
619
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai11
A Unified Approach to Interpreting Model Predictions2017
"Why Should I Trust You?": Explaining the Predictions of Any Classifier2016
Distilling the Knowledge in a Neural Network2015
Causality: Models, Reasoning, and Inference2009
Towards A Rigorous Science of Interpretable Machine Learning2017
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)2018
Causability and explainability of artificial intelligence in medicine2019
Can we open the black box of AI?2016
Machine Learning Interpretability: A Survey on Methods and Metrics2019
Interactive machine learning for health informatics: when do we need the human-in-the-loop?2016
A Survey of Methods for Explaining Black Box Models2018
Cited by5
Explainable artificial intelligence (XAI) in finance: a systematic literature review2024
From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI2023
Explainable Artificial Intelligence (XAI) 2.0: A manifesto of open challenges and interdisciplinary research directions2024
Transparency of deep neural networks for medical image analysis: A review of interpretability methods2021
A Survey on Explainable Anomaly Detection2023