MoCo-v2

Last updated on Feb 27, 2021

MoCo-v2 ResNet-50 (200 epochs, 256 bs)

MoCo-v2 ResNet-50 (200 epochs, 256 bs) achieves 83.2% Top 1 Accuracy on ImageNet


Parameters 26 Million
FLOPs 4 Billion
File Size 352.86 MB
Training Data ImageNet
Training Resources 8x NVIDIA V100 GPUs
Training Time

Training Techniques MoCo v2, Weight Decay, SGD with Momentum
Architecture 1x1 Convolution, Bottleneck Residual Block, Batch Normalization, Convolution, Global Average Pooling, Residual Block, Residual Connection, ReLU, Max Pooling, Softmax
ID rn50_moco_in1k_moco_style
LR 0.03
Epochs 200
Layers 50
Momentum 0.9
Batch Size 256
Weight Decay 0.0001
Width Multiplier 1
MOCO Loss Temperature 0.2
SHOW MORE
SHOW LESS
README.md

Summary

MoCo v2 is an improved version of the Momentum Contrast self-supervised learning algorithm. Improvements include:

  • Replacing the 1-layer fully connected layer with a 2-layer MLP head.
  • Including blur augmentation (the same used in SimCLR).

How do I train this model?

Get started with VISSL by trying one of the Colab tutorial notebooks.

Citation

@misc{chen2020improved,
      title={Improved Baselines with Momentum Contrastive Learning}, 
      author={Xinlei Chen and Haoqi Fan and Ross Girshick and Kaiming He},
      year={2020},
      eprint={2003.04297},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Results

Image Classification on ImageNet

Image Classification
BENCHMARK MODEL METRIC NAME METRIC VALUE GLOBAL RANK
ImageNet MoCo-v2 ResNet-50 (200 epochs, 256 bs) Top 1 Accuracy 66.4% # 291