A Convolutional Encoder Model for Neural Machine Translation
The prevalent approach to neural machine translation relies on bi-directional LSTMs to encode the source sentence. In this paper we present a faster and simpler architecture based on a succession of convolutional layers. This allows to encode the entire source sentence simultaneously compared to recurrent networks for which computation is constrained by temporal dependencies. On WMT'16 English-Romanian translation we achieve competitive accuracy to the state-of-the-art and we outperform several recently published results on the WMT'15 English-German task. Our models obtain almost the same accuracy as a very deep LSTM setup on WMT'14 English-French translation. Our convolutional encoder speeds up CPU decoding by more than two times at the same or higher accuracy as a strong bi-directional LSTM baseline.
PDF Abstract ACL 2017 PDF ACL 2017 AbstractCode
Datasets
WMT 2014
WMT 2016
WMT 2016 News
Results from the Paper
| Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Benchmark |
|---|---|---|---|---|---|---|
| Machine Translation | IWSLT2015 German-English | Conv-LSTM (deep+pos) | BLEU score | 30.4 | # 7 | |
| Machine Translation | WMT2014 English-French | Deep Convolutional Encoder; single-layer decoder | BLEU score | 35.7 | # 45 | |
| Machine Translation | WMT2016 English-Romanian | BiLSTM | BLEU score | 27.5 | # 15 | |
| Machine Translation | WMT2016 English-Romanian | Deep Convolutional Encoder; single-layer decoder | BLEU score | 27.8 | # 14 |
