Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
Обучение представлений фраз с использованием RNN Encoder–Decoder для статистического машинного перевода
2014-06-03
SCID: 54.1/63njxgy3
Discuss with AI
RNN Encoder-Decoderconditional phrase probabilityphrase representationrecurrent neural networkstatistical machine translation
Figures from the paper
Abstract (AI)
In this paper, we propose a novel neural network model called RNN Encoder-Decoder that consists of two recurrent neural networks (RNN). One RNN encodes a sequence of symbols into a fixed-length vector representation, and the other decodes the representation into another sequence of symbols. The encoder and decoder of the proposed model are jointly trained to maximize the conditional probability of a target sequence given a source sequence. The performance of a statistical machine translation system is empirically found to improve by using the conditional probabilities of phrase pairs computed by the RNN Encoder-Decoder as an additional feature in the existing log-linear model. Qualitatively, we show that the proposed model learns a semantically and syntactically meaningful representation of linguistic phrases.
Key Findings
1
Encoder and decoder are jointly trained to maximize the conditional probability of a target sequence given a source sequence.
2
Introduced the RNN Encoder-Decoder architecture composed of two RNNs: one encodes a symbol sequence to a fixed-length vector, the other decodes it to another sequence.
3
The model learns semantically and syntactically meaningful representations of linguistic phrases, demonstrated qualitatively.
4
Using RNN Encoder-Decoder–computed conditional probabilities of phrase pairs as an additional feature improves statistical machine translation performance.
Research Object
RNN Encoder-Decoder model for learning phrase representations (two recurrent neural networks: encoder and decoder)
Research Subject
Learning fixed-length vector representations of phrase sequences and using the model's conditional probabilities of phrase pairs to improve statistical machine translation performance
Publication Details
Publication Date
2014-06-03
Journal
Publisher
ISSN
Cited by
5374
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai2
Cited by18
Sequence to Sequence Learning with Neural Networks2014
Neural Machine Translation by Jointly Learning to Align and Translate2014
Representation Learning with Contrastive Predictive Coding2018
A Review of Recurrent Neural Networks: LSTM Cells and Network Architectures2019
Understanding and Mitigating Gradient Flow Pathologies in Physics-Informed Neural Networks2021
CodeBERT: A Pre-Trained Model for Programming and Natural Languages2020
Self-Attention with Relative Position Representations2018
A Survey of Machine and Deep Learning Methods for Internet of Things (IoT) Security2020
MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding2020
A Unifying Review of Deep and Shallow Anomaly Detection2021
AI applications to medical images: From machine learning to deep learning2021
A Survey of Deep Learning Methods for Cyber Security2019
Modeling Coverage for Neural Machine Translation2016
Traffic Flow Prediction via Spatial Temporal Graph Neural Network2020
Deep Learning for Medical Anomaly Detection – A Survey2021
A Survey on Machine Learning Techniques for Routing Optimization in SDN2021
Recent Advances in Anomaly Detection Methods Applied to Aviation2019
Log Parsing with Generalization Ability under New Log Types2023