Recurrent Neural Networks: A Comprehensive Review of Architectures, Variants, and Applications
Рекуррентные нейронные сети: всесторонний обзор архитектур, вариантов и приложений
2024-08-25
SCID: 54.1/uqq38f8x
Discuss with AI
attention mechanismsgated recurrent unitslong short-term memoryrecurrent neural networkstime series forecasting
Figures from the paper
Abstract (AI)
Recurrent neural networks (RNNs) have significantly advanced the field of machine learning (ML) by enabling the effective processing of sequential data. This paper provides a comprehensive review of RNNs and their applications, highlighting advancements in architectures, such as long short-term memory (LSTM) networks, gated recurrent units (GRUs), bidirectional LSTM (BiLSTM), echo state networks (ESNs), peephole LSTM, and stacked LSTM. The study examines the application of RNNs to different domains, including natural language processing (NLP), speech recognition, time series forecasting, autonomous vehicles, and anomaly detection. Additionally, the study discusses recent innovations, such as the integration of attention mechanisms and the development of hybrid models that combine RNNs with convolutional neural networks (CNNs) and transformer architectures. This review aims to provide ML researchers and practitioners with a comprehensive overview of the current state and future directions of RNN research.
Key Findings
1
RNNs are applied across diverse domains, including NLP, speech recognition, time series forecasting, autonomous vehicles, and anomaly detection.
2
RNNs enable effective processing of sequential data and have substantially advanced machine learning applications.
3
Recent developments include integrating attention mechanisms and combining RNNs with CNNs or transformer architectures in hybrid models.
4
The review covers major RNN variants, including LSTM, GRU, BiLSTM, ESN, peephole LSTM, and stacked LSTM architectures.
5
The review identifies current RNN research directions and summarizes their evolving architectures and applications.
Research Object
recurrent neural networks (RNNs) and their architectural variants
Research Subject
architectures, variants, applications, and recent innovations of RNNs for sequential-data processing
Publication Details
Publication Date
2024-08-25
Journal
Publisher
ISSN
Cited by
983
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai13
Gradient-based learning applied to document recognition1998
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation2014
Neural Machine Translation by Jointly Learning to Align and Translate2014
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling2014
Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups2012
Effective Approaches to Attention-based Neural Machine Translation2015
A Review of Recurrent Neural Networks: LSTM Cells and Network Architectures2019
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation2016
Deep learning modelling techniques: current progress, applications, advantages, and challenges2023
DeepAnT: A Deep Learning Approach for Unsupervised Anomaly Detection in Time Series2018
Time-Series Anomaly Detection Service at Microsoft2019
A Deep Learning Ensemble With Data Resampling for Credit Card Fraud Detection2023
Modern computing: Vision and challenges2024