CoMatch: Semi-supervised Learning with Contrastive Graph Regularization

ICCV 2021  ·  Junnan Li, Caiming Xiong, Steven Hoi ·

Semi-supervised learning has been an effective paradigm for leveraging unlabeled data to reduce the reliance on labeled data. We propose CoMatch, a new semi-supervised learning method that unifies dominant approaches and addresses their limitations. CoMatch jointly learns two representations of the training data, their class probabilities and low-dimensional embeddings. The two representations interact with each other to jointly evolve. The embeddings impose a smoothness constraint on the class probabilities to improve the pseudo-labels, whereas the pseudo-labels regularize the structure of the embeddings through graph-based contrastive learning. CoMatch achieves state-of-the-art performance on multiple datasets. It achieves substantial accuracy improvements on the label-scarce CIFAR-10 and STL-10. On ImageNet with 1% labels, CoMatch achieves a top-1 accuracy of 66.0%, outperforming FixMatch by 12.6%. Furthermore, CoMatch achieves better representation learning performance on downstream tasks, outperforming both supervised learning and self-supervised learning. Code and pre-trained models are available at https://github.com/salesforce/CoMatch.

PDF Abstract ICCV 2021 PDF ICCV 2021 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Semi-Supervised Image Classification CIFAR-10, 20 Labels CoMatch (SimCLR) Percentage error 12.33±8.47 # 2
Semi-Supervised Image Classification CIFAR-10, 40 Labels CoMatch (w. SimCLR) Percentage error 6.91±1.39 # 13
Semi-Supervised Image Classification CIFAR-10, 80 Labels SimCLR (CoMatch) Percentage error 5.98 # 2
Semi-Supervised Image Classification ImageNet - 10% labeled data CoMatch (w. MoCo v2) Top 5 Accuracy 91.4% # 12
Top 1 Accuracy 73.7% # 28
Semi-Supervised Image Classification ImageNet - 1% labeled data CoMatch (w. MoCo v2) Top 5 Accuracy 87.1% # 12
Top 1 Accuracy 67.1% # 21
Semi-Supervised Image Classification STL-10, 1000 Labels SimCLR (CoMatch) Accuracy 77.46 # 9

Methods