Meta-Embeddings Based On Self-Attention

no code implementations • 3 Mar 2020 • Qichen Li, Yuanqing Lin, Luofeng Zhou, Jian Li

Creating meta-embeddings for better performance in language modelling has received attention lately, and methods based on concatenation or merely calculating the arithmetic mean of more than one separately trained embeddings to perform meta-embeddings have shown to be beneficial.

Language Modelling Machine Translation +3

SUM: Suboptimal Unitary Multi-task Learning Framework for Spatiotemporal Data Prediction

no code implementations • 11 Oct 2019 • Qichen Li, Jiaxin Pei, Jianding Zhang, Bo Han

However, such a method have relatively weak performance when the task number is small, and we cannot integrate it into non-linear models.

Meta-Learning Multi-Task Learning