Search Results for author:
Found 6 papers, 4 papers with code
Mixture Data for Training Cannot Ensure Out-of-distribution Generalization
Deep neural networks often face generalization problems to handle out-of-distribution (OOD) data, and there remains a notable theoretical gap between the contributing factors and their respective impacts.
Revisiting Knowledge Distillation under Distribution Shift
Knowledge distillation transfers knowledge from large models into small models, and has recently made remarkable achievements.
A Quality-based Syntactic Template Retriever for Syntactically-controlled Paraphrase Generation
Furthermore, for situations requiring multiple paraphrases for each source sentence, we design a Diverse Templates Search (DTS) algorithm, which can enhance the diversity between paraphrases without sacrificing quality.
Towards Understanding and Improving Knowledge Distillation for Neural Machine Translation
In this work, we first unravel this mystery from an empirical perspective and show that the knowledge comes from the top-1 predictions of teachers, which also helps us build a potential connection between word- and sequence-level KD.
Improved Data Augmentation for Translation Suggestion
This paper introduces the system used in our submission to the WMT'22 Translation Suggestion shared task.
Conditional Bilingual Mutual Information Based Adaptive Training for Neural Machine Translation
Token-level adaptive training approaches can alleviate the token imbalance problem and thus improve neural machine translation, through re-weighting the losses of different target tokens based on specific statistical metrics (e. g., token frequency or mutual information).
