Abstract
Neural machine translation is a recently proposed approach to machine translation. Unlike the traditional statistical machine translation, the neural machine translation aims at building a single neural network that can be jointly tuned to maximize the translation performance. The models proposed recently for neural machine translation often belong to a family of encoder-decoders and consists of an encoder that encodes a source sentence into a fixed-length vector from which a decoder generates a translation. In this paper, we conjecture that the use of a fixed-length vector is a bottleneck in improving the performance of this basic encoder-decoder architecture, and propose to extend this by allowing a model to automatically (soft-)search for parts of a source sentence that are relevant to predicting a target word, without having to form these parts as a hard segment explicitly. With this new approach, we achieve a translation performance comparable to the existing state-of-the-art phrase-based system on the task of English-to-French translation. Furthermore, qualitative analysis reveals that the (soft-)alignments found by the model agree well with our intuition.
Keywords
Machine translationComputer scienceTransfer-based machine translationExample-based machine translationSentenceBottleneckArtificial intelligenceTranslation (biology)Artificial neural networkNatural language processingEncoderPhraseWord (group theory)Speech recognition
Affiliated Institutions
Related Publications
Attention Is All You Need
The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also co...
UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation
The state-of-the-art models for medical image segmentation are variants of U-Net and fully convolutional networks (FCN). Despite their success, these models have two limitations...
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Spatial pyramid pooling module or encode-decoder structure are used in deep neural networks for semantic segmentation task. The former networks are able to encode multi-scale co...
UNet++: A Nested U-Net Architecture for Medical Image Segmentation
In this paper, we present UNet++, a new, more powerful architecture for medical image segmentation. Our architecture is essentially a deeply-supervised encoder-decoder network w...
Low-density parity-check codes
A low-density parity-check code is a code specified by a parity-check matrix with the following properties: each column contains a small fixed number <tex xmlns:mml="http://www....
Publication Info
- Year
- 2014
- Type
- preprint
- Citations
- 14564
- Access
- Closed
External Links
Social Impact
Altmetric
PlumX Metrics
Social media, news, blog, policy document mentions
Citation Metrics
14564
OpenAlex
Cite This
Dzmitry Bahdanau,
Kyunghyun Cho,
Yoshua Bengio
(2014).
Neural Machine Translation by Jointly Learning to Align and Translate.
arXiv (Cornell University)
.
https://doi.org/10.48550/arxiv.1409.0473
Identifiers
- DOI
- 10.48550/arxiv.1409.0473