Self-Supervised Learning
1734 papers with code • 10 benchmarks • 41 datasets
Self-Supervised Learning is proposed for utilizing unlabeled data with the success of supervised learning. Producing a dataset with good labels is expensive, while unlabeled data is being generated all the time. The motivation of Self-Supervised Learning is to make use of the large amount of unlabeled data. The main idea of Self-Supervised Learning is to generate the labels from unlabeled data, according to the structure or characteristics of the data itself, and then train on this unsupervised data in a supervised manner. Self-Supervised Learning is wildly used in representation learning to make a model learn the latent features of the data. This technique is often employed in computer vision, video processing and robot control.
Source: Self-supervised Point Set Local Descriptors for Point Cloud Registration
Image source: LeCun
Benchmarks
These leaderboards are used to track progress in Self-Supervised Learning
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
Libraries
Use these libraries to find Self-Supervised Learning models and implementationsDatasets
CIFAR-10
CIFAR-100
STL-10
Tiny ImageNet
2D-3D-S
NSynth
AVA
CATER
MotionSense
COCO 10% labeled data
Latest papers with no code
An Effective Automated Speaking Assessment Approach to Mitigating Data Scarcity and Imbalanced Distribution
Automated speaking assessment (ASA) typically involves automatic speech recognition (ASR) and hand-crafted feature extraction from the ASR transcript of a learner's speech.
Mitigating Object Dependencies: Improving Point Cloud Self-Supervised Learning through Object Exchange
Subsequently, we introduce a context-aware feature learning strategy, which encodes object patterns without relying on their specific context by aggregating object features across various scenes.
Encoding Urban Ecologies: Automated Building Archetype Generation through Self-Supervised Learning for Energy Modeling
As the global population and urbanization expand, the building sector has emerged as the predominant energy consumer and carbon emission contributor.
LaTiM: Longitudinal representation learning in continuous-time models to predict disease progression
This work proposes a novel framework for analyzing disease progression using time-aware neural ordinary differential equations (NODE).
How to Craft Backdoors with Unlabeled Data Alone?
Relying only on unlabeled data, Self-supervised learning (SSL) can learn rich features in an economical and scalable way.
Wild Visual Navigation: Fast Traversability Learning via Pre-Trained Models and Online Self-Supervision
Natural environments such as forests and grasslands are challenging for robotic navigation because of the false perception of rigid obstacles from high grass, twigs, or bushes.
Anomaly Detection in Electrocardiograms: Advancing Clinical Diagnosis Through Self-Supervised Learning
We introduce a novel self-supervised learning framework for ECG AD, utilizing a vast dataset of normal ECGs to autonomously detect and localize cardiac anomalies.
HDR Imaging for Dynamic Scenes with Events
High dynamic range imaging (HDRI) for real-world dynamic scenes is challenging because moving objects may lead to hybrid degradation of low dynamic range and motion blur.
Multi-modal Learning for WebAssembly Reverse Engineering
WasmRev is pre-trained using self-supervised learning on a large-scale multi-modal corpus encompassing source code, code documentation and the compiled WebAssembly, without requiring labeled data.
Generative-Contrastive Heterogeneous Graph Neural Network
In recent years, inspired by self-supervised learning, contrastive Heterogeneous Graphs Neural Networks (HGNNs) have shown great potential by utilizing data augmentation and discriminators for downstream tasks.
