CoVe, or Contextualized Word Vectors, uses a deep LSTM encoder from an attentional sequence-to-sequence model trained for machine translation to contextualize word vectors. $\text{CoVe}$ word embeddings are therefore a function of the entire input sequence. These word embeddings can then be used in downstream tasks by concatenating them with $\text{GloVe}$ embeddings:
$$ v = \left[\text{GloVe}\left(x\right), \text{CoVe}\left(x\right)\right]$$
and then feeding these in as features for the task-specific models.
Source: Learned in Translation: Contextualized Word VectorsPaper | Code | Results | Date | Stars |
---|
Task | Papers | Share |
---|---|---|
Retrieval | 3 | 10.00% |
Translation | 3 | 10.00% |
Text Generation | 2 | 6.67% |
Reading Comprehension | 2 | 6.67% |
Language Modelling | 2 | 6.67% |
Machine Translation | 2 | 6.67% |
Question Answering | 2 | 6.67% |
Click-Through Rate Prediction | 1 | 3.33% |
Collaborative Filtering | 1 | 3.33% |
Component | Type |
|
---|---|---|
BiLSTM
|
Deep Tabular Learning | |
GloVe
|
Word Embeddings | |
Location-based Attention
|
Attention Mechanisms | |
Seq2Seq
|
Sequence To Sequence Models | |
Softmax
|
Output Functions |