A Review of Recurrent Neural Networks: LSTM Cells and Network Architectures
Обзор рекуррентных нейронных сетей: LSTM‑ячейки и архитектуры сетей
2019-05-21
SCID: 54.1/6canrfqp
Discuss with AI
LSTM cellLSTM variantsLSTM-dominated networkslong short-term memoryrecurrent neural networks
Figures from the paper
Abstract (AI)
Recurrent neural networks (RNNs) have been widely adopted in research areas concerned with sequential data, such as text, audio, and video. However, RNNs consisting of sigma cells or tanh cells are unable to learn the relevant information of input data when the input gap is large. By introducing gate functions into the cell structure, the long short-term memory (LSTM) could handle the problem of long-term dependencies well. Since its introduction, almost all the exciting results based on RNNs have been achieved by the LSTM. The LSTM has become the focus of deep learning. We review the LSTM cell and its variants to explore the learning capacity of the LSTM cell. Furthermore, the LSTM networks are divided into two broad categories: LSTM-dominated networks and integrated LSTM networks. In addition, their various applications are discussed. Finally, future research directions are presented for LSTM networks.
Key Findings
1
Almost all notable RNN-based results since LSTM's introduction have been achieved using LSTM architectures, making LSTM central to deep learning progress on sequential data.
2
Introducing gate functions in LSTM cells enables effective handling of long-term dependencies compared to traditional RNN cells.
3
LSTM networks can be categorized into two broad classes—LSTM-dominated networks and integrated LSTM networks—and their diverse applications and future research directions are discussed.
4
Standard RNNs with sigma or tanh cells struggle to learn relevant information when input gaps are large, failing at long-term dependencies.
5
The paper reviews LSTM cell variants to analyze and explore the LSTM's learning capacity.
Research Object
Long Short-Term Memory (LSTM) cells and LSTM-based recurrent neural network architectures
Research Subject
The learning capacity, variants, categorizations (LSTM-dominated vs integrated LSTM networks), applications, and future research directions of LSTM cells and LSTM-based networks, especially their ability to handle long-term dependencies in sequential data
Publication Details
Publication Date
2019-05-21
Journal
Publisher
ISSN
Cited by
6551
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai4
Cited by12
Deep learning modelling techniques: current progress, applications, advantages, and challenges2023
Recurrent Neural Networks: A Comprehensive Review of Architectures, Variants, and Applications2024
Financial Fraud: A Review of Anomaly Detection Techniques and Recent Advances2021
Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A Survey2023
Machine Learning Methods for Small Data Challenges in Molecular Science2023
Generative artificial intelligence in drug discovery: basic framework, recent advances, challenges, and opportunities2024
A Deep Learning Ensemble With Data Resampling for Credit Card Fraud Detection2023
Large Language Models for Forecasting and Anomaly Detection: A Systematic Literature Review2024
Large Language Models: A Structured Taxonomy and Review of Challenges, Limitations, Solutions, and Future Directions2025
Comparative Analysis of Deep Natural Networks and Large Language Models for Aspect-Based Sentiment Analysis2024
Deep Learning for Sustainable Aquaculture: Opportunities and Challenges2025
State-of-the-Art Wayside Condition Monitoring Systems for Railway Wheels: A Comprehensive Review2023