We introduce COLA, a self-supervised pre-training approach for learning a general-purpose representation of audio.
Contrastive unsupervised learning has recently shown encouraging progress, e. g., in Momentum Contrast (MoCo) and SimCLR.
Ranked #18 on Self-Supervised Image Classification on ImageNet
This enables building a large and consistent dictionary on-the-fly that facilitates contrastive unsupervised learning.
Ranked #22 on Self-Supervised Image Classification on ImageNet
In this work, we formulate cross-lingual language model pre-training as maximizing mutual information between multilingual-multi-granularity texts.
This paper presents SimCLR: a simple framework for contrastive learning of visual representations.
However, designing these views requires considerable human expertise and experimentation, hindering widespread adoption of unsupervised representation learning methods across domains and modalities.
Contrastive learning has recently shown immense potential in unsupervised visual representation learning.
We analyze key properties of the approach that make it work, finding that the contrastive loss outperforms a popular alternative based on cross-view prediction, and that the more views we learn from, the better the resulting representation captures underlying scene semantics.
Ranked #18 on Self-Supervised Action Recognition on UCF101
Contrastive learning between multiple views of the data has recently achieved state of the art performance in the field of self-supervised representation learning.
Ranked #11 on Self-Supervised Image Classification on ImageNet