Large language models (LLMs): survey, technical frameworks, and future challenges
Большие языковые модели (LLM): обзор, технические основы и будущие проблемы
2024-08-18
SCID: 54.1/gmyabm5r
Discuss with AI
code generationlanguage modeling architectureslarge language modelsnatural language processingvision-language models
Figures from the paper
Abstract (AI)
Artificial intelligence (AI) has significantly impacted various fields. Large language models (LLMs) like GPT-4, BARD, PaLM, Megatron-Turing NLG, Jurassic-1 Jumbo etc., have contributed to our understanding and application of AI in these domains, along with natural language processing (NLP) techniques. This work provides a comprehensive overview of LLMs in the context of language modeling, word embeddings, and deep learning. It examines the application of LLMs in diverse fields including text generation, vision-language models, personalized learning, biomedicine, and code generation. The paper offers a detailed introduction and background on LLMs, facilitating a clear understanding of their fundamental ideas and concepts. Key language modeling architectures are also discussed, alongside a survey of recent works employing LLM methods for various downstream tasks across different domains. Additionally, it assesses the limitations of current approaches and highlights the need for new methodologies and potential directions for significant advancements in this field.
Key Findings
1
It identifies limitations of current LLM approaches and outlines directions for developing new methodologies and advancing the field.
2
It reviews major LLM architectures and surveys their use in downstream tasks across text generation, vision-language modeling, personalized learning, biomedicine, and code generation.
3
The paper provides a comprehensive survey of large language models, covering language modeling, word embeddings, and deep learning foundations.
4
The survey discusses representative models including GPT-4, BARD, PaLM, Megatron-Turing NLG, and Jurassic-1 Jumbo.
Research Object
Large language models (LLMs)
Research Subject
their architectures, applications across domains, downstream-task performance, limitations, and future methodological directions
Publication Details
Publication Date
2024-08-18
Journal
Publisher
ISSN
Cited by
387
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai16
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation2014
LLaMA: Open and Efficient Foundation Language Models2023
Enriching Word Vectors with Subword Information2017
Machine learning: Trends, perspectives, and prospects2015
Effective Approaches to Attention-based Neural Machine Translation2015
Hierarchical Attention Networks for Document Classification2016
Transformer-XL: Attentive Language Models beyond a Fixed-Length Context2019
ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns2023
Attention mechanisms in computer vision: A survey2022
Self-Attention with Relative Position Representations2018
BioGPT: generative pre-trained transformer for biomedical text generation and mining2022
AI-Based Modeling: Techniques, Applications and Research Issues Towards Automation, Intelligent and Smart Systems2022
Efficient Transformers: A Survey2022
Introducing Artificial Intelligence Training in Medical Education2019
Human- versus Artificial Intelligence2021
Are We There Yet? - A Systematic Literature Review on Chatbots in Education2021