Interpretable machine learning: Fundamental principles and 10 grand challenges
Интерпретируемое машинное обучение: основные принципы и 10 крупных задач
2022-01-01
SCID: 54.1/bpq64gzr
Discuss with AI
Rashomon setcase-based reasoningcausal constraintscausal inference matchingdata visualizationdecision treesdimensionality reductiondisentanglementgeneralized additive modelsgenerative constraintsinterpretable machine learninginterpretable reinforcement learningphysics-informed machine learningscoring systemssparse logical modelsunsupervised disentanglement
Figures from the paper
Abstract (AI)
Interpretability in machine learning (ML) is crucial for high stakes decisions and troubleshooting. In this work, we provide fundamental principles for interpretable ML, and dispel common misunderstandings that dilute the importance of this crucial topic. We also identify 10 technical challenge areas in interpretable machine learning and provide history and background on each problem. Some of these problems are classically important, and some are recent problems that have arisen in the last few years. These problems are: (1) Optimizing sparse logical models such as decision trees; (2) Optimization of scoring systems; (3) Placing constraints into generalized additive models to encourage sparsity and better interpretability; (4) Modern case-based reasoning, including neural networks and matching for causal inference; (5) Complete supervised disentanglement of neural networks; (6) Complete or even partial unsupervised disentanglement of neural networks; (7) Dimensionality reduction for data visualization; (8) Machine learning models that can incorporate physics and other generative or causal constraints; (9) Characterization of the “Rashomon set” of good models; and (10) Interpretable reinforcement learning. This survey is suitable as a starting point for statisticians and computer scientists interested in working in interpretable machine learning.
Key Findings
1
It identifies ten specific technical challenge areas in interpretable ML, spanning classic and recent problems.
2
The paper presents fundamental principles for interpretable machine learning and clarifies common misunderstandings that undermine its importance.
3
The survey provides history and background on each identified problem and is intended as a starting point for statisticians and computer scientists entering interpretable ML.
4
The ten challenge areas are: optimizing sparse logical models (e.g., decision trees); optimizing scoring systems; adding constraints to generalized additive models for sparsity and interpretability; modern case-based reasoning including neural networks and causal matching; complete supervised disentanglement of neural networks; complete or partial unsupervised disentanglement of neural networks; dimensionality reduction for data visualization; incorporating physics or generative/causal constraints into ML models; characterizing the Rashomon set of good models; and interpretable reinforcement learning.
Research Object
Interpretable machine learning (the field and its models/practices)
Research Subject
Fundamental principles, common misunderstandings, and ten grand technical challenge areas in interpretable ML (including optimization of sparse logical models, scoring systems, constrained GAMs, case-based reasoning, supervised and unsupervised disentanglement, visualization, physics/causal constraints, Rashomon set characterization, and interpretable reinforcement learning)
Publication Details
Publication Date
2022-01-01
Journal
Publisher
ISSN
Cited by
954
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai10
ImageNet classification with deep convolutional neural networks2017
Scikit-learn: Machine Learning in Python2012
Greedy function approximation: A gradient boosting machine.2001
Representation Learning: A Review and New Perspectives2013
Understanding and Mitigating Gradient Flow Pathologies in Physics-Informed Neural Networks2021
Reinforcement learning in robotics: A survey2013
Accelerating t-SNE using tree-based algorithms2014
The art of using t-SNE for single-cell transcriptomics2019
Trustworthy artificial intelligence2020
Understanding How Dimension Reduction Tools Work: An Empirical Approach to Deciphering t-SNE, UMAP, TriMAP, and PaCMAP for Data Visualization2020
Cited by6
Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence2023
Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next2022
Explainable Artificial Intelligence (XAI): What we know and what is left to attain Trustworthy Artificial Intelligence2023
Explainable AI for 6G Use Cases: Technical Aspects and Research Challenges2024
Artificial intelligence for climate prediction of extremes: State of the art, challenges, and future perspectives2024
Artificial intelligence-driven real-world battery diagnostics2024