Search Results for author:
Found 26 papers, 11 papers with code
Non-autoregressive Translation with Disentangled Context Transformer
State-of-the-art neural machine translation models generate a translation from left to right and every step is conditioned on the previously generated tokens.
Ranked #54 on
Machine Translation
on WMT2014 English-German
Facebook AI’s WMT21 News Translation Task Submission
We describe Facebook’s multilingual model submission to the WMT2021 shared task on news translation.
Efficiently Upgrading Multilingual Machine Translation Models to Support More Languages
With multilingual machine translation (MMT) models continuing to grow in size and number of supported languages, it is natural to reuse and upgrade existing models to save computation as data becomes available in more languages.
No Language Left Behind: Scaling Human-Centered Machine Translation
Driven by the goal of eradicating language barriers on a global scale, machine translation has solidified itself as a key focus of artificial intelligence research today.
Ranked #1 on
Machine Translation
on IWSLT2017 French-English
(SacreBLEU metric)
Multilingual Neural Machine Translation with Deep Encoder and Multiple Shallow Decoders
Recent work in multilingual translation advances translation quality surpassing bilingual baselines using deep transformer models with increased capacity.
Multilingual Machine Translation with Hyper-Adapters
We find that hyper-adapters are more parameter efficient than regular adapters, reaching the same performance with up to 12 times less parameters.
Lifting the Curse of Multilinguality by Pre-training Modular Transformers
Multilingual pre-trained models are known to suffer from the curse of multilinguality, which causes per-language performance to drop as they cover more languages.
How Robust is Neural Machine Translation to Language Imbalance in Multilingual Tokenizer Training?
Since a skewed data distribution is considered to be harmful, a sampling strategy is usually used to balance languages in the corpus.
Data Selection Curriculum for Neural Machine Translation
In this work, we introduce a two-stage curriculum training framework for NMT where we fine-tune a base NMT model on subsets of data, selected by both deterministic scoring using pre-trained methods and online scoring that considers prediction scores of the emerging NMT model.
Alternative Input Signals Ease Transfer in Multilingual Machine Translation
Further, we find that incorporating alternative inputs via self-ensemble can be particularly effective when training set is small, leading to +5 BLEU when only 5% of the total training data is accessible.
Tricks for Training Sparse Translation Models
Multi-task learning with an unbalanced data distribution skews model learning towards high resource tasks, especially when model capacity is fixed and fully shared across all tasks.
Classification-based Quality Estimation: Small and Efficient Models for Real-world Applications
Sentence-level Quality estimation (QE) of machine translation is traditionally formulated as a regression task, and the performance of QE models is typically measured by Pearson correlation with human labels.
Facebook AI WMT21 News Translation Task Submission
We describe Facebook's multilingual model submission to the WMT2021 shared task on news translation.
On the Evaluation of Machine Translation for Terminology Consistency
As neural machine translation (NMT) systems become an important part of professional translator pipelines, a growing body of work focuses on combining NMT with terminologies.
XLEnt: Mining a Large Cross-lingual Entity Dataset with Lexical-Semantic-Phonetic Word Alignment
Cross-lingual named-entity lexica are an important resource to multilingual NLP tasks such as machine translation and cross-lingual wikification.
Improving Zero-Shot Translation by Disentangling Positional Information
The difficulty of generalizing to new translation directions suggests the model representations are highly specific to those language pairs seen in training.
Learn to Talk via Proactive Knowledge Transfer
In this paper, we provide an in-depth analysis of KL-divergence minimization in Forward and Backward orders, which shows that learners are reinforced via on-policy learning in Backward.
Deep Encoder, Shallow Decoder: Reevaluating Non-autoregressive Machine Translation
We show that the speed disadvantage for autoregressive baselines compared to non-autoregressive methods has been overestimated in three aspects: suboptimal layer allocation, insufficient speed measurement, and lack of knowledge distillation.
Non-Autoregressive Machine Translation with Disentangled Context Transformer
State-of-the-art neural machine translation models generate a translation from left to right and every step is conditioned on the previously generated tokens.
Monotonic Multihead Attention
Simultaneous machine translation models start generating a target sequence before they have encoded or read the source sequence.
Proactive Sequence Generator via Knowledge Acquisition
Sequence-to-sequence models such as transformers, which are now being used in a wide variety of NLP tasks, typically need to have very high capacity in order to perform well.
Simple Fusion: Return of the Language Model
Neural Machine Translation (NMT) typically leverages monolingual data in training through backtranslation.
Span-Based Constituency Parsing with a Structure-Label System and Provably Optimal Dynamic Oracles
Parsing accuracy using efficient greedy transition systems has improved dramatically in recent years thanks to neural networks.
Incremental Parsing with Minimal Features Using Bi-Directional LSTM
Recently, neural network approaches for parsing have largely automated the combination of individual features, but still rely on (often a larger number of) atomic features created from human linguistic intuition, and potentially omitting important global context.
Good, Better, Best: Choosing Word Embedding Context
We propose two methods of learning vector representations of words and phrases that each combine sentence context with structural features extracted from dependency trees.
