Unsupervised Data Augmentation for Consistency Training

arXiv 2019 Qizhe XieZihang DaiEduard HovyMinh-Thang LuongQuoc V. Le

Semi-supervised learning lately has shown much promise in improving deep learning models when labeled data is scarce. Common among recent approaches is the use of consistency training on a large amount of unlabeled data to constrain model predictions to be invariant to input noise... (read more)

PDF Abstract
TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK COMPARE
Text Classification Amazon-2 BERT Finetune + UDA Error 3.50 # 2
Text Classification Amazon-5 BERT Finetune + UDA Error 37.12 # 2
Semi-Supervised Image Classification CIFAR-10, 4000 Labels UDA Accuracy 97.3 # 1
Semi-Supervised Image Classification ImageNet - 10% labeled data UDA Top 5 Accuracy 88.52% # 2
Text Classification IMDb BERT Finetune + UDA Accuracy 95.80 # 2
Semi-Supervised Image Classification SVHN, 1000 labels UDA Accuracy 97.54 # 1
Text Classification Yelp-2 BERT Finetune + UDA Accuracy 97.95% # 2
Text Classification Yelp-5 BERT Finetune + UDA Accuracy 67.92% # 2