GPT (Generative Pre-Trained Transformer)— A Comprehensive Review on Enabling Technologies, Potential Applications, Emerging Challenges, and Future Directions
GPT (генеративно предварительно обученный трансформер): всесторонний обзор обеспечивающих технологий, потенциальных применений, возникающих проблем и перспективных направлений
2024-01-01
SCID: 54.1/jg84bcun
Discuss with AI
Deep neural networksGPTGenerative Pre-trained TransformerNatural language processingTransformer architecture
Figures from the paper
Abstract (AI)
The Generative Pre-trained Transformer (GPT) represents a notable breakthrough in the domain of natural language processing, which is propelling us toward the development of machines that can understand and communicate using language in a manner that closely resembles that of humans. GPT is based on the transformer architecture, a deep neural network designed for natural language processing tasks. Due to their impressive performance on natural language processing tasks and ability to effectively converse, GPT have gained significant popularity among researchers and industrial communities, making them one of the most widely used and effective models in natural language processing and related fields, which motivated to conduct this review. This review provides a detailed overview of the GPT, including its architecture, working process, training procedures, enabling technologies, and its impact on various applications. In this review, we also explored the potential challenges and limitations of a GPT. Furthermore, we discuss potential solutions and future directions. Overall, this paper aims to provide a comprehensive understanding of GPT, its enabling technologies, their impact on various applications, emerging challenges, and potential solutions.
Key Findings
1
GPT demonstrates strong performance across natural language processing tasks and effective conversational capabilities, driving widespread research and industrial adoption.
2
GPT, based on the Transformer architecture, is identified as a major advancement toward human-like language understanding and communication.
3
The paper examines GPT’s potential applications, emerging challenges, and limitations, and discusses proposed solutions and future research directions.
4
The review synthesizes GPT’s architecture, operating process, training procedures, and enabling technologies, along with their effects on diverse applications.
Research Object
Generative Pre-trained Transformer (GPT) models
Research Subject
GPT architecture, operation, training procedures, enabling technologies, applications, challenges, limitations, and future directions
Publication Details
Publication Date
2024-01-01
Journal
Publisher
ISSN
Cited by
603
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai16
Exploiting Generative AI to Scale up Intelligent Tutoring Systems2023
Deep Learning with Differential Privacy2016
Opinion Paper: “So what if ChatGPT wrote it?” Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy2023
Education in the Era of Generative Artificial Intelligence (AI): Understanding the Potential Benefits of ChatGPT in Promoting Teaching and Learning2023
GPT-3: Its Nature, Scope, Limits, and Consequences2020
On the Opportunities and Risks of Foundation Models2021
ChatGPT: A comprehensive review on background, applications, key challenges, bias, ethics, limitations and future scope2023
The rise of deep learning in drug discovery2018
Sparks of Artificial General Intelligence: Early experiments with GPT-42023
ChatGPT and a new academic reality: Artificial Intelligence‐written research papers and the ethics of the large language models in scholarly publishing2023
A Survey on the Role of IoT in Agriculture for the Implementation of Smart Farming2019
The AI gambit: leveraging artificial intelligence to combat climate change—opportunities, challenges, and recommendations2021
Edge Computing with Artificial Intelligence: A Machine Learning Perspective2022
GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models2023
A Review of ChatGPT AI's Impact on Several Business Sectors2023
Chat2VIS: Generating Data Visualizations via Natural Language Using ChatGPT, Codex and GPT-3 Large Language Models2023