Large Language Models: A Comprehensive Survey of its Applications, Challenges, Limitations, and Future Prospects
Большие языковые модели: всесторонний обзор их приложений, проблем, ограничений и перспектив развития
2023-11-15
SCID: 54.1/42rp2hzq
Discuss with AI
generative pre-trained transformerslarge language modelsmachine translationmodel biasnatural language processing
Figures from the paper
Abstract (AI)
Within the vast expanse of computerized language processing, a revolutionary entity known as Large Language Models (LLMs) has emerged, wielding immense power in its capacity to comprehend intricate linguistic patterns and conjure coherent and contextually fitting responses. Large language models (LLMs) are a type of artificial intelligence (AI) that have emerged as powerful tools for a wide range of tasks, including natural language processing (NLP), machine translation, and question-answering. This survey paper provides a comprehensive overview of LLMs, including their history, architecture, training methods, applications, and challenges. The paper begins by discussing the fundamental concepts of generative AI and the architecture of generative pre- trained transformers (GPT). It then provides an overview of the history of LLMs, their evolution over time, and the different training methods that have been used to train them. The paper then discusses the wide range of applications of LLMs, including medical, education, finance, and engineering. It also discusses how LLMs are shaping the future of AI and how they can be used to solve real-world problems. The paper then discusses the challenges associated with deploying LLMs in real-world scenarios, including ethical considerations, model biases, interpretability, and computational resource requirements. It also highlights techniques for enhancing the robustness and controllability of LLMs, and addressing bias, fairness, and generation quality issues. Finally, the paper concludes by highlighting the future of LLM research and the challenges that need to be addressed in order to make LLMs more reliable and useful. This survey paper is intended to provide researchers, practitioners, and enthusiasts with a comprehensive understanding of LLMs, their evolution, applications, and challenges. By consolidating the state-of-the-art knowledge in the field, this survey serves as a valuable resource for further advancements in the development and utilization of LLMs for a wide range of real-world applications. The GitHub repo for this project is available at https://github.com/anas-zafar/LLM-Survey
Key Findings
1
LLMs have broad application potential across medicine, education, finance, and engineering, including solutions for real-world problems.
2
Large language models have become powerful general-purpose AI tools for natural language processing, machine translation, and question answering.
3
Real-world deployment is constrained by ethical concerns, model bias, limited interpretability, computational requirements, and generation-quality issues.
4
Robustness, controllability, bias mitigation, fairness, and generation-quality techniques are identified as important directions for making LLMs more reliable and useful.
5
The survey synthesizes LLM history, GPT-based architectures, evolution, and training methods to provide a consolidated overview of the field.
Research Object
large language models (LLMs)
Research Subject
their architectures, training methods, applications, deployment challenges, limitations, robustness, controllability, bias, fairness, interpretability, and generation quality
Publication Details
Publication Date
2023-11-15
Journal
Publisher
ISSN
Cited by
193
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai20
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer2019
LLaMA: Open and Efficient Foundation Language Models2023
Generative adversarial networks2020
Transformers: State-of-the-Art Natural Language Processing2020
A Survey of Large Language Models2026
Object Detection With Deep Learning: A Review2019
Recurrent neural network based language model2010
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation2016
Large language models encode clinical knowledge2023
Deep Reinforcement Learning: A Brief Survey2017
Generative Adversarial Networks: An Overview2018
Training language models to follow instructions with human feedback2022
Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models2023
Robust Speech Recognition via Large-Scale Weak Supervision2022
Transformer-XL: Attentive Language Models beyond a Fixed-Length Context2019
A survey on large language model based autonomous agents2024
A Comprehensive Overview of Large Language Models2025
ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns2023
HuggingFace's Transformers: State-of-the-art Natural Language Processing2019
GPT-3: Its Nature, Scope, Limits, and Consequences2020
Cited by5
A Comprehensive Overview of Large Language Models2025
A Review on Large Language Models: Architectures, Applications, Taxonomies, Open Issues and Challenges2024
Large Language Models: A Structured Taxonomy and Review of Challenges, Limitations, Solutions, and Future Directions2025
The role of agentic artificial intelligence in healthcare: a scoping review2026
LearnRAG: Implementing Retrieval-Augmented Generation for Adaptive Learning Systems2025