Improved Baselines with Momentum Contrastive Learning
Contrastive unsupervised learning has recently shown encouraging progress, e.g., in Momentum Contrast (MoCo) and SimCLR. In this note, we verify the effectiveness of two of SimCLR's design improvements by implementing them in the MoCo framework. With simple modifications to MoCo---namely, using an MLP projection head and more data augmentation---we establish stronger baselines that outperform SimCLR and do not require large training batches. We hope this will make state-of-the-art unsupervised learning research more accessible. Code will be made public.
PDF AbstractCode
Colab
Datasets
ImageNet
Places205
SYSU-30k
Methods
1x1 Convolution •
Average Pooling •
Batch Normalization •
Bottleneck Residual Block •
Convolution •
Cosine Annealing •
Dense Connections •
Feedforward Network •
Global Average Pooling •
InfoNCE •
Kaiming Initialization •
Max Pooling •
MoCo •
MoCo v2 •
Random Gaussian Blur •
Random Horizontal Flip •
Random Resized Crop •
ReLU •
Residual Block •
Residual Connection •
ResNet •
SGD with Momentum
