Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Эмпирическая оценка управляемых рекуррентных нейронных сетей для моделирования последовательностей
2014-12-11
SCID: 54.1/fjm3hw38
Discuss with AI
gated recurrent unit (GRU)long short-term memory (LSTM)polyphonic music modelingrecurrent neural networks (RNNs)speech signal modeling
Figures from the paper
Abstract (AI)
In this paper we compare different types of recurrent units in recurrent neural networks (RNNs). Especially, we focus on more sophisticated units that implement a gating mechanism, such as a long short-term memory (LSTM) unit and a recently proposed gated recurrent unit (GRU). We evaluate these recurrent units on the tasks of polyphonic music modeling and speech signal modeling. Our experiments revealed that these advanced recurrent units are indeed better than more traditional recurrent units such as tanh units. Also, we found GRU to be comparable to LSTM.
Key Findings
1
Evaluations were conducted on polyphonic music modeling and speech signal modeling benchmarks.
2
GRU performs comparably to LSTM in empirical evaluations.
3
Gated recurrent units (LSTM and GRU) outperform traditional tanh recurrent units on sequence modeling tasks.
Research Object
Recurrent neural network recurrent units (LSTM, GRU, tanh units) evaluated on sequence modeling tasks
Research Subject
Comparative empirical performance of gated vs. traditional recurrent units (LSTM, GRU, tanh) on sequence modeling tasks, specifically polyphonic music modeling and speech signal modeling
Publication Details
Publication Date
2014-12-11
Journal
Publisher
ISSN
Cited by
13337
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai4
Cited by20
Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting2021
A Review of Recurrent Neural Networks: LSTM Cells and Network Architectures2019
Automatic Chemical Design Using a Data-Driven Continuous Representation of Molecules2018
Attention Based Spatial-Temporal Graph Convolutional Networks for Traffic Flow Forecasting2019
Deep Learning for Hyperspectral Image Classification: An Overview2019
Robust Anomaly Detection for Multivariate Time Series through Stochastic Recurrent Neural Network2019
Review of Deep Learning Algorithms and Architectures2019
Tacotron: Towards End-to-End Speech Synthesis2017
Deep Learning for IoT Big Data and Streaming Analytics: A Survey2018
Graph Neural Networks in Recommender Systems: A Survey2022
Network intrusion detection system: A systematic study of machine learning and deep learning approaches2020
Machine Learning and Deep Learning Methods for Intrusion Detection Systems: A Survey2019
Machine learning & artificial intelligence in the quantum domain: a review of recent progress2018
Recurrent Neural Networks: A Comprehensive Review of Architectures, Variants, and Applications2024
Deep Learning for Time Series Anomaly Detection: A Survey2024
Multivariate Time-Series Anomaly Detection via Graph Attention Network2020
Modeling Coverage for Neural Machine Translation2016
Multivariate Time Series Anomaly Detection and Interpretation using Hierarchical Inter-Metric and Temporal Embedding2021
A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detection2024
Physics-informed machine learning: A comprehensive review on applications in anomaly detection and condition monitoring2024