Search Results for author:
Found 7 papers, 3 papers with code
Learning Evaluation Models from Large Language Models for Sequence Generation
Large language models achieve state-of-the-art performance on sequence generation evaluation, but typically have a large number of parameters.
ODE Transformer: An Ordinary Differential Equation-Inspired Model for Sequence Generation
Inspired by this, we design a new architecture, {\it ODE Transformer}, which is analogous to the Runge-Kutta method that is well motivated in ODE.
ODE Transformer: An Ordinary Differential Equation-Inspired Model for Neural Machine Translation
We show that a residual block of layers in Transformer can be described as a higher-order solution to ODEs.
Learning Light-Weight Translation Models from Deep Transformer
We proposed a novel group-permutation based knowledge distillation approach to compressing the deep Transformer model into a shallow model.
A Simple and Effective Approach to Robust Unsupervised Bilingual Dictionary Induction
Our experiments show that this simple method does not hamper the performance of similar language pairs and achieves an accuracy of 13. 64~55. 53% between English and four distant languages, i. e., Chinese, Japanese, Vietnamese and Thai.
Shallow-to-Deep Training for Neural Machine Translation
We find that stacking layers is helpful in improving the representation ability of NMT models and adjacent layers perform similarly.
Weight Distillation: Transferring the Knowledge in Neural Network Parameters
Inspired by this, we investigate methods of model acceleration and compression in another line of research.
