Semi-Supervised Sequence Modeling with Cross-View Training

EMNLP 2018 Kevin ClarkMinh-Thang LuongChristopher D. ManningQuoc V. Le

Unsupervised representation learning algorithms such as word2vec and ELMo improve the accuracy of many supervised NLP models, mainly because they can take advantage of large amounts of unlabeled text. However, the supervised models only learn from task-specific labeled data during the main training phase... (read more)

PDF Abstract

Results from the Paper


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK USES EXTRA
TRAINING DATA
RESULT BENCHMARK
CCG Supertagging CCGBank CVT + Multi-task + Large Accuracy 96.1 # 1
Named Entity Recognition CoNLL 2003 (English) CVT + Multi-Task F1 92.61 # 12
Named Entity Recognition CoNLL 2003 (English) CVT + Multi-Task + Large F1 92.61 # 12
Machine Translation IWSLT2015 English-Vietnamese CVT BLEU 29.6 # 4
Named Entity Recognition Ontonotes v5 (English) CVT + Multi-Task + Large F1 88.81 # 7
Dependency Parsing Penn Treebank CVT + Multi-Task + Large UAS 96.61 # 1
LAS 95.02 # 1
Part-Of-Speech Tagging Penn Treebank CVT + Multi-task Accuracy 97.76 # 4
Dependency Parsing Penn Treebank CVT + Multi-Task UAS 96.44 # 2
LAS 94.83 # 2

Methods used in the Paper