Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results

NeurIPS 2017 Antti TarvainenHarri Valpola

The recently proposed Temporal Ensembling has achieved state-of-the-art results in several semi-supervised learning benchmarks. It maintains an exponential moving average of label predictions on each training example, and penalizes predictions that are inconsistent with this target... (read more)

PDF Abstract
TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT LEADERBOARD
Semi-Supervised Image Classification CIFAR-10, 250 Labels MeanTeacher Accuracy 52.68 # 7
Semi-Supervised Image Classification CIFAR-10, 4000 Labels Mean Teacher Accuracy 93.72 # 7
Semi-Supervised Image Classification ImageNet - 10% labeled data Mean Teacher (ResNeXt-152) Top 5 Accuracy 90.89% # 9
Semi-Supervised Image Classification SVHN, 1000 labels Mean Teacher Accuracy 96.05 # 8
Semi-Supervised Image Classification SVHN, 250 Labels MeanTeacher Accuracy 93.55 # 5

Methods used in the Paper