A Survey of Methods for Explaining Black Box Models
Обзор методов объяснения решений моделей типа «чёрный ящик»
2018-08-22
SCID: 54.1/tyk3wkvd
Discuss with AI
black box modelsdecision support systemsexplainable artificial intelligenceexplanation methodsmodel interpretability
Figures from the paper
Abstract (AI)
In recent years, many accurate decision support systems have been constructed as black boxes, that is as systems that hide their internal logic to the user. This lack of explanation constitutes both a practical and an ethical issue. The literature reports many approaches aimed at overcoming this crucial weakness, sometimes at the cost of sacrificing accuracy for interpretability. The applications in which black box decision systems can be used are various, and each approach is typically developed to provide a solution for a specific problem and, as a consequence, it explicitly or implicitly delineates its own definition of interpretability and explanation. The aim of this article is to provide a classification of the main problems addressed in the literature with respect to the notion of explanation and the type of black box system. Given a problem definition, a black box type, and a desired explanation, this survey should help the researcher to find the proposals more useful for his own work. The proposed classification of approaches to open black box models should also be useful for putting the many research open questions in perspective.
Key Findings
1
Black-box decision systems create practical and ethical problems because they hide their internal logic from users.
2
Existing explanation methods may improve interpretability but can require sacrificing some predictive accuracy.
3
Explanation approaches differ according to the application, black-box system type, and the specific notion of interpretability they adopt.
4
The proposed classification organizes outstanding research questions concerning the explanation and opening of black-box models.
5
The survey classifies major research problems and methods by black-box type and desired explanation to guide method selection.
Research Object
Black-box decision support systems (black box models)
Research Subject
methods and approaches for explaining their internal logic and classifying explanations by problem, black-box type, and desired interpretability
Publication Details
Publication Date
2018-08-22
Journal
Publisher
ISSN
Cited by
629
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai5
Cited by20
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)2018
Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence2023
Explainable Artificial Intelligence (XAI): What we know and what is left to attain Trustworthy Artificial Intelligence2023
Explainable Machine Learning for Scientific Insights and Discoveries2020
Bias in data‐driven artificial intelligence systems—An introductory survey2020
Future of industry 5.0 in society: human-centric solutions, challenges and prospective research areas2022
A Survey on the Explainability of Supervised Machine Learning2021
Questioning the AI: Informing Design Practices for Explainable AI User Experiences2020
From What to How: An Initial Review of Publicly Available AI Ethics Tools, Methods and Research to Translate Principles into Practices2019
AI applications to medical images: From machine learning to deep learning2021
Attention in Natural Language Processing2020
Explainable artificial intelligence (XAI) in finance: a systematic literature review2024
Evaluating the Quality of Machine Learning Explanations: A Survey on Methods and Metrics2021
From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI2023
Explainable Artificial Intelligence (XAI) 2.0: A manifesto of open challenges and interdisciplinary research directions2024
Explainable AI for Healthcare 5.0: Opportunities and Challenges2022
Explainable, trustworthy, and ethical machine learning for healthcare: A survey2022
Explainable Artificial Intelligence in CyberSecurity: A Survey2022
A multilayer multimodal detection and prediction model based on explainable artificial intelligence for Alzheimer’s disease2021
Survey of explainable artificial intelligence techniques for biomedical imaging with deep neural networks2023