Unsupervised Data Augmentation for Consistency Training

ICLR 2020 Qizhe XieZihang DaiEduard HovyMinh-Thang LuongQuoc V. Le

Semi-supervised learning lately has shown much promise in improving deep learning models when labeled data is scarce. Common among recent approaches is the use of consistency training on a large amount of unlabeled data to constrain model predictions to be invariant to input noise... (read more)

PDF Abstract

Results from the Paper


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT LEADERBOARD
Text Classification Amazon-2 BERT Finetune + UDA Error 3.5 # 2
Text Classification Amazon-5 BERT Finetune + UDA Error 37.12 # 3
Semi-Supervised Image Classification CIFAR-10, 4000 Labels UDA Accuracy 94.73 # 5
Image Classification ImageNet ResNet-50 (UDA) Top 1 Accuracy 79.04% # 73
Semi-Supervised Image Classification ImageNet - 10% labeled data UDA Top 5 Accuracy 88.52% # 12
Text Classification IMDb BERT Finetune + UDA Accuracy 95.8 # 2
Semi-Supervised Image Classification SVHN, 1000 labels UDA Accuracy 97.54 # 3
Text Classification Yelp-2 BERT Finetune + UDA Accuracy 97.95% # 3
Text Classification Yelp-5 BERT Finetune + UDA Accuracy 67.92% # 5

Methods used in the Paper