A Survey on Large Language Models: Applications, Challenges, Limitations, and Practical Usage
Обзор больших языковых моделей: приложения, проблемы, ограничения и практическое применение
2023-07-10
SCID: 54.1/39ysyvse
Discuss with AI
generative pre-trained transformerslarge language modelsmachine translationmodel biasnatural language processing
Figures from the paper
Abstract (AI)
Within the vast expanse of computerized language processing, a revolutionary entity known as Large Language Models (LLMs) has emerged, wielding immense power in its capacity to comprehend intricate linguistic patterns and conjure coherent and contextually fitting responses. Large language models (LLMs) are a type of artificial intelligence (AI) that have emerged as powerful tools for a wide range of tasks, including natural language processing (NLP), machine translation, and question-answering. This survey paper provides a comprehensive overview of LLMs, including their history, architecture, training methods, applications, and challenges. The paper begins by discussing the fundamental concepts of generative AI and the architecture of generative pre- trained transformers (GPT). It then provides an overview of the history of LLMs, their evolution over time, and the different training methods that have been used to train them. The paper then discusses the wide range of applications of LLMs, including medical, education, finance, and engineering. It also discusses how LLMs are shaping the future of AI and how they can be used to solve real-world problems. The paper then discusses the challenges associated with deploying LLMs in real-world scenarios, including ethical considerations, model biases, interpretability, and computational resource requirements. It also highlights techniques for enhancing the robustness and controllability of LLMs, and addressing bias, fairness, and generation quality issues. Finally, the paper concludes by highlighting the future of LLM research and the challenges that need to be addressed in order to make LLMs more reliable and useful. This survey paper is intended to provide researchers, practitioners, and enthusiasts with a comprehensive understanding of LLMs, their evolution, applications, and challenges. By consolidating the state-of-the-art knowledge in the field, this survey serves as a valuable resource for further advancements in the development and utilization of LLMs for a wide range of real-world applications. The GitHub repo for this project is available at https://github.com/anas-zafar/LLM-Survey
Key Findings
1
Future LLM research must address reliability and usefulness to enable broader real-world problem solving.
2
LLMs support diverse applications across natural language processing, machine translation, question answering, medicine, education, finance, and engineering.
3
Real-world LLM deployment faces ethical, bias, interpretability, and computational-resource challenges.
4
The survey highlights techniques for improving LLM robustness, controllability, fairness, bias mitigation, and generation quality.
5
The survey reviews large language models’ history, architectures, and training methods, including generative AI and GPT-based transformers.
Research Object
Large Language Models (LLMs)
Research Subject
The history, architectures, training methods, applications, deployment challenges, limitations, and techniques for improving the robustness, controllability, fairness, interpretability, and generation quality of LLMs
Publication Details
Publication Date
2023-07-10
Journal
Publisher
ISSN
Cited by
785
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai20
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer2019
LLaMA: Open and Efficient Foundation Language Models2023
Transformers: State-of-the-Art Natural Language Processing2020
A Survey of Large Language Models2026
Opinion Paper: “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy2023
Recurrent neural network based language model2010
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation2016
Generative Adversarial Networks: An Overview2018
Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models2023
ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns2023
GPT-3: Its Nature, Scope, Limits, and Consequences2020
PaLM: Scaling Language Modeling with Pathways2022
ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope2023
BloombergGPT: A Large Language Model for Finance2023
How Does ChatGPT Perform on the United States Medical Licensing Examination (USMLE)? The Implications of Large Language Models for Medical Education and Knowledge Assessment2023
Artificial intelligence in higher education: the state of the field2023
Sparks of Artificial General Intelligence: Early experiments with GPT-42023
Evaluating Large Language Models Trained on Code2021
Examining Science Education in ChatGPT: An Exploratory Study of Generative Artificial Intelligence2023
ChatGPT and a new academic reality: Artificial Intelligence‐written research papers and the ethics of the large language models in scholarly publishing2023
Cited by3
A survey on large language model (LLM) security and privacy: The Good, The Bad, and The Ugly2024
A Review on Large Language Models: Architectures, Applications, Taxonomies, Open Issues and Challenges2024
Transforming Education: A Comprehensive Review of Generative Artificial Intelligence in Educational Settings through Bibliometric and Content Analysis2023