Explainability for Large Language Models: A Survey
Объяснимость больших языковых моделей: обзор
2024-01-02
SCID: 54.1/2xh8g3ma
Discuss with AI
Transformer-based language modelsexplainability techniquesexplanation evaluation metricslarge language modelslocal and global explanations
Figures from the paper
Abstract (AI)
Large language models (LLMs) have demonstrated impressive capabilities in natural language processing. However, their internal mechanisms are still unclear and this lack of transparency poses unwanted risks for downstream applications. Therefore, understanding and explaining these models is crucial for elucidating their behaviors, limitations, and social impacts. In this article, we introduce a taxonomy of explainability techniques and provide a structured overview of methods for explaining Transformer-based language models. We categorize techniques based on the training paradigms of LLMs: traditional fine-tuning-based paradigm and prompting-based paradigm. For each paradigm, we summarize the goals and dominant approaches for generating local explanations of individual predictions and global explanations of overall model knowledge. We also discuss metrics for evaluating generated explanations and discuss how explanations can be leveraged to debug models and improve performance. Lastly, we examine key challenges and emerging opportunities for explanation techniques in the era of LLMs in comparison to conventional deep learning models.
Key Findings
1
For each paradigm, the survey distinguishes local explanations of individual predictions from global explanations of overall model knowledge.
2
It identifies key challenges and emerging opportunities for explaining LLMs compared with conventional deep learning models.
3
It organizes explanation methods according to two LLM training paradigms: traditional fine-tuning and prompting.
4
The survey introduces a taxonomy of explainability techniques for Transformer-based large language models.
5
The survey reviews metrics for evaluating generated explanations and describes how explanations can support model debugging and performance improvement.
Research Object
large language models (LLMs), particularly Transformer-based language models
Research Subject
explainability of their internal mechanisms, behaviors, limitations, social impacts, predictions, and learned knowledge
Publication Details
Publication Date
2024-01-02
Journal
Publisher
ISSN
Cited by
675
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai9
A Unified Approach to Interpreting Model Predictions2017
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer2019
"Why Should I Trust You?": Explaining the Predictions of Any Classifier2016
LLaMA: Open and Efficient Foundation Language Models2023
Proceedings of the 24th international conference on Machine learning2007
Towards A Rigorous Science of Interpretable Machine Learning2017
Lost in the Middle: How Language Models Use Long Contexts2024
🧜Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Models2025
Evaluating the Effectiveness of Large Language Models in Representing Textual Descriptions of Geometry and Spatial Relations (Short Paper)2023
Cited by5
A Comprehensive Overview of Large Language Models2025
Retrieval-augmented generation for generative artificial intelligence in health care2025
Materials science in the era of large language models: a perspective2024
Generative AI and LLMs for Critical Infrastructure Protection: Evaluation Benchmarks, Agentic AI, Challenges, and Opportunities2025
A Research Landscape of Agentic AI and Large Language Models: Applications, Challenges and Future Directions2025