Meta Pseudo Labels

We present Meta Pseudo Labels, a semi-supervised learning method that achieves a new state-of-the-art top-1 accuracy of 90.2% on ImageNet, which is 1.6% better than the existing state-of-the-art. Like Pseudo Labels, Meta Pseudo Labels has a teacher network to generate pseudo labels on unlabeled data to teach a student network. However, unlike Pseudo Labels where the teacher is fixed, the teacher in Meta Pseudo Labels is constantly adapted by the feedback of the student's performance on the labeled dataset. As a result, the teacher generates better pseudo labels to teach the student. Our code will be available at https://github.com/google-research/google-research/tree/master/meta_pseudo_labels.

PDF Abstract CVPR 2021 PDF CVPR 2021 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Semi-Supervised Image Classification CIFAR-10, 4000 Labels Meta Pseudo Labels (WRN-28-2) Percentage error 3.89± 0.07 # 2
Image Classification ImageNet Meta Pseudo Labels (EfficientNet-L2) Top 1 Accuracy 90.2% # 13
Number of params 480M # 935
Hardware Burden 95040G # 1
Operations per network pass None # 1
Image Classification ImageNet Meta Pseudo Labels (EfficientNet-B6-Wide) Top 1 Accuracy 90% # 19
Number of params 390M # 929
Image Classification ImageNet Meta Pseudo Labels (ResNet-50) Top 1 Accuracy 83.2% # 413
Semi-Supervised Image Classification ImageNet - 10% labeled data Meta Pseudo Labels (ResNet-50) Top 5 Accuracy 91.38% # 13
Top 1 Accuracy 73.89% # 27
Image Classification ImageNet ReaL Meta Pseudo Labels (EfficientNet-B6-Wide) Accuracy 91.12% # 4
Image Classification ImageNet ReaL Meta Pseudo Labels (EfficientNet-L2) Accuracy 91.02% # 8
Semi-Supervised Image Classification SVHN, 1000 labels Meta Pseudo Labels (WRN-28-2) Accuracy 98.01 ± 0.07 # 1

Methods