On the Properties of Neural Machine Translation: Encoder–Decoder Approaches
О свойствах нейронного машинного перевода: подходы энкодер–декодер
2014-01-01
SCID: 54.1/t9za7h8r
Discuss with AI
RNN Encoder-Decoderencoder–decoderfixed-length representationgated recursive convolutional neural networkneural machine translation
Figures from the paper
Abstract (AI)
Neural machine translation is a relatively new approach to statistical machine translation based purely on neural networks.The neural machine translation models often consist of an encoder and a decoder.The encoder extracts a fixed-length representation from a variable-length input sentence, and the decoder generates a correct translation from this representation.In this paper, we focus on analyzing the properties of the neural machine translation using two models; RNN Encoder-Decoder and a newly proposed gated recursive convolutional neural network.We show that the neural machine translation performs relatively well on short sentences without unknown words, but its performance degrades rapidly as the length of the sentence and the number of unknown words increase.Furthermore, we find that the proposed gated recursive convolutional network learns a grammatical structure of a sentence automatically.
Key Findings
1
A newly proposed gated recursive convolutional neural network (grConv) can be used as an encoder alternative to RNN Encoder-Decoder.
2
NMT performance degrades rapidly as sentence length increases and as the number of unknown words increases.
3
NMT performs relatively well on short sentences without unknown words.
4
Neural machine translation (NMT) models typically use an encoder to produce a fixed-length representation and a decoder to generate translations.
5
The gated recursive convolutional network automatically learns grammatical structure of sentences.
Research Object
Neural machine translation models (encoder–decoder architectures including RNN Encoder-Decoder and gated recursive convolutional neural network)
Research Subject
Properties and performance characteristics of encoder–decoder neural machine translation, including sentence-length and unknown-word sensitivity and the gated recursive convolutional network's ability to learn grammatical structure
Publication Details
Publication Date
2014-01-01
Journal
Publisher
ISSN
Cited by
8614
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai2
Cited by11
Neural Machine Translation by Jointly Learning to Align and Translate2014
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling2014
Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting2021
Deep learning for healthcare: review, opportunities and challenges2017
Deep Learning for Time Series Anomaly Detection: A Survey2024
Modeling Coverage for Neural Machine Translation2016
A review on sentiment analysis from social media platforms2023
Hyperspectral Image Classification—Traditional to Deep Models: A Survey for Future Prospects2021
Computing Graph Neural Networks: A Survey from Algorithms to Accelerators2021
RenAIssance: A Survey Into AI Text-to-Image Generation in the Era of Large Model2024
A Comparative Deep Learning Framework for Multivariate Time Series Anomaly Detection in Satellite Telemetry2026