A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models
Обзор взаимодействия RAG и LLM: к большим языковым моделям с дополнением извлечённой информацией
2024-08-24
SCID: 54.1/hht4fbn7
Discuss with AI
Large Language ModelsRetrieval-Augmented GenerationRetrieval-Augmented Large Language Modelsexternal knowledge baseshallucinations
Figures from the paper
Abstract (AI)
As one of the most advanced techniques in AI, Retrieval-Augmented Generation (RAG) can offer reliable and up-to-date external knowledge, providing huge convenience for numerous tasks. Particularly in the era of AI-Generated Content (AIGC), the powerful capacity of retrieval in providing additional knowledge enables RAG to assist existing generative AI in producing high-quality outputs. Recently, Large Language Models (LLMs) have demonstrated revolutionary abilities in language understanding and generation, while still facing inherent limitations such as hallucinations and out-of-date internal knowledge. Given the powerful abilities of RAG in providing the latest and helpful auxiliary information, Retrieval-Augmented Large Language Models (RA-LLMs) have emerged to harness external and authoritative knowledge bases, rather than solely relying on the model's internal knowledge, to augment the quality of the generated content of LLMs. In this survey, we comprehensively review existing research studies in RA-LLMs, covering three primary technical perspectives: Furthermore, to deliver deeper insights, we discuss current limitations and several promising directions for future research. Updated information about this survey can be found at: https://advanced-recommender-systems.github.io/RAG-Meets-LLMs/
Key Findings
1
Retrieval-Augmented Generation supplies external, reliable, and up-to-date knowledge to improve the quality of large language model outputs.
2
Retrieval-Augmented Large Language Models address LLM limitations including hallucinations and outdated internal knowledge by incorporating authoritative external knowledge bases.
3
The survey comprehensively organizes existing RA-LLM research across three primary technical perspectives.
4
The survey identifies current limitations of RA-LLMs and outlines promising directions for future research.
Research Object
Retrieval-Augmented Large Language Models (RA-LLMs)
Research Subject
The use of retrieval from external authoritative knowledge bases to augment LLM-generated content, including the technical perspectives, limitations, and future research directions of RA-LLMs
Publication Details
Publication Date
2024-08-24
Journal
Publisher
ISSN
Cited by
943
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai18
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer2019
A Survey of Large Language Models2026
Dense Passage Retrieval for Open-Domain Question Answering2020
Retrieval-Augmented Generation for Large Language Models: A Survey2023
PaLM: Scaling Language Modeling with Pathways2022
GPT-4 Technical Report2023
Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering2021
Evaluating Large Language Models Trained on Code2021
Recommender Systems in the Era of Large Language Models (LLMs)2024
Active Retrieval Augmented Generation2023
Atlas: Few-shot Learning with Retrieval Augmented Language Models2022
Retrieval-Augmented Generation for AI-Generated Content: A Survey2026
Retrieval Augmentation Reduces Hallucination in Conversation2021
In-Context Retrieval-Augmented Language Models2023
Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answering2023
PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models2024
Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy2023
Enhancing Financial Sentiment Analysis via Retrieval Augmented Large Language Models2023
Cited by5
A Survey of Large Language Models2026
Retrieval-Augmented Generation for AI-Generated Content: A Survey2026
Knowledge Graph-Guided Retrieval Augmented Generation2025
SafeRAG: Benchmarking Security in Retrieval-Augmented Generation of Large Language Model2025
SpaRAGraph: Spatial Reasoning using Retrieval-Augmented Generation2026