Retrieval-Augmented Generation for Large Language Models: A Survey
Генерация с дополнением извлечённой информацией для больших языковых моделей: обзор
2023-12-18
SCID: 54.1/h883evpx
Discuss with AI
External Knowledge RetrievalKnowledge-Intensive TasksLarge Language ModelsRAG Evaluation BenchmarksRetrieval-Augmented Generation
Figures from the paper
Abstract (AI)
Large Language Models (LLMs) showcase impressive capabilities but encounter challenges like hallucination, outdated knowledge, and non-transparent, untraceable reasoning processes. Retrieval-Augmented Generation (RAG) has emerged as a promising solution by incorporating knowledge from external databases. This enhances the accuracy and credibility of the generation, particularly for knowledge-intensive tasks, and allows for continuous knowledge updates and integration of domain-specific information. RAG synergistically merges LLMs' intrinsic knowledge with the vast, dynamic repositories of external databases. This comprehensive review paper offers a detailed examination of the progression of RAG paradigms, encompassing the Naive RAG, the Advanced RAG, and the Modular RAG. It meticulously scrutinizes the tripartite foundation of RAG frameworks, which includes the retrieval, the generation and the augmentation techniques. The paper highlights the state-of-the-art technologies embedded in each of these critical components, providing a profound understanding of the advancements in RAG systems. Furthermore, this paper introduces up-to-date evaluation framework and benchmark. At the end, this article delineates the challenges currently faced and points out prospective avenues for research and development.
Key Findings
1
External knowledge retrieval improves generation accuracy and credibility, especially for knowledge-intensive tasks, while enabling continuous updates and domain-specific information integration.
2
RAG frameworks are analyzed through three core components: retrieval, generation, and augmentation, including state-of-the-art techniques for each.
3
Retrieval-Augmented Generation (RAG) addresses LLM hallucinations, outdated knowledge, and untraceable reasoning by incorporating information from external databases.
4
The paper presents an up-to-date evaluation framework and benchmark, and identifies current challenges and future research directions for RAG systems.
5
The survey organizes RAG development into Naive RAG, Advanced RAG, and Modular RAG paradigms.
Research Object
Retrieval-Augmented Generation (RAG) systems for Large Language Models
Research Subject
RAG paradigms, architectures, component technologies, evaluation, challenges, and future research directions
Publication Details
Publication Date
2023-12-18
Journal
Publisher
ISSN
Cited by
8625
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai14
Dense Passage Retrieval for Open-Domain Question Answering2020
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank2013
Lost in the Middle: How Language Models Use Long Contexts2024
🧜Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Models2025
Scaling Laws for Neural Language Models2020
Active Retrieval Augmented Generation2023
Atlas: Few-shot Learning with Retrieval Augmented Language Models2022
Benchmarking Large Language Models in Retrieval-Augmented Generation2024
Chatlaw: A Multi-Agent Legal Assistant based on a Role-Aligned Mixture-of-Experts Architecture2026
In-Context Retrieval-Augmented Language Models2023
The Power of Noise: Redefining Retrieval for RAG Systems2024
Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answering2023
Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy2023
CRUD-RAG: A Comprehensive Chinese Benchmark for Retrieval-Augmented Generation of Large Language Models2024
Cited by20
A Survey of Large Language Models2026
Retrieval-Augmented Generation for AI-Generated Content: A Survey2026
A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models2024
Benchmarking Retrieval-Augmented Generation for Medicine2024
LightRAG: Simple and Fast Retrieval-Augmented Generation2025
MM-LLMs: Recent Advances in MultiModal Large Language Models2024
Chatlaw: A Multi-Agent Legal Assistant based on a Role-Aligned Mixture-of-Experts Architecture2026
Large Language Models for Cyber Security: A Systematic Literature Review2025
Graph Retrieval-Augmented Generation: A Survey2025
AgentClinic: a multimodal benchmark for tool-using clinical AI agents2026
Knowledge Graph-Guided Retrieval Augmented Generation2025
Retrieval augmented generation for large language models in healthcare: A systematic review2025
A Survey on Large Language Models for Code Generation2025
Improving large language model applications in biomedicine with retrieval-augmented generation: a systematic review, meta-analysis, and clinical development guidelines2025
CRUD-RAG: A Comprehensive Chinese Benchmark for Retrieval-Augmented Generation of Large Language Models2024
Materials science in the era of large language models: a perspective2024
Clinical entity augmented retrieval for clinical information extraction2025
A survey on retrieval-augmentation generation (RAG) models for healthcare applications2025
Retrieval-Augmented Generation (RAG)2025
The effects of generative AI agents and scaffolding on enhancing students’ comprehension of visual learning analytics2025