Modeling Coverage for Neural Machine Translation
Моделирование покрытия для нейронного машинного перевода
2016-01-01
SCID: 54.1/yhqwak6b
Discuss with AI
alignment qualityattention mechanismcoverage vectorneural machine translationover-translation and under-translation
Figures from the paper
Abstract (AI)
Attention mechanism has enhanced stateof-the-art Neural Machine Translation (NMT) by jointly learning to align and translate.It tends to ignore past alignment information, however, which often leads to over-translation and under-translation.To address this problem, we propose coverage-based NMT in this paper.We maintain a coverage vector to keep track of the attention history.The coverage vector is fed to the attention model to help adjust future attention, which lets NMT system to consider more about untranslated source words.Experiments show that the proposed approach significantly improves both translation quality and alignment quality over standard attention-based NMT. 1
Key Findings
1
Feeding the coverage vector into the attention model adjusts future attention to reduce translation errors.
2
Introducing a coverage vector that tracks attention history helps the model consider untranslated source words.
3
Standard attention-based NMT ignores past alignment information, causing over-translation and under-translation.
4
The coverage-based NMT significantly improves both translation quality and alignment quality compared to standard attention-based NMT.
Research Object
Attention-based Neural Machine Translation system (with coverage vector)
Research Subject
Modeling and using a coverage vector to track attention history for reducing over-translation and under-translation, improving translation and alignment quality
Publication Details
Publication Date
2016-01-01
Journal
Publisher
ISSN
Cited by
715
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai8
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation2014
Sequence to Sequence Learning with Neural Networks2014
Neural Machine Translation by Jointly Learning to Align and Translate2014
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling2014
Bidirectional recurrent neural networks1997
On the Properties of Neural Machine Translation: Encoder–Decoder Approaches2014
Effective Approaches to Attention-based Neural Machine Translation2015
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation2014