Unsupervised Pre-training
104 papers with code • 2 benchmarks • 7 datasets
Pre-training a neural network using unsupervised (self-supervised) auxiliary tasks on unlabeled data.
Benchmarks
These leaderboards are used to track progress in Unsupervised Pre-training
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
Libraries
Use these libraries to find Unsupervised Pre-training models and implementations
CARLANE Benchmark
m2caiSeg
GBUSV
TYC Dataset
Icon645
Most implemented papers
Unsupervised Semantic Segmentation by Contrasting Object Mask Proposals
wvangansbeke/Unsupervised-Semantic-Segmentation
•
•
ICCV 2021
To achieve this, we introduce a two-step framework that adopts a predetermined mid-level prior in a contrastive optimization objective to learn pixel embeddings.
DOBF: A Deobfuscation Pre-Training Objective for Programming Languages
Recent advances in self-supervised learning have dramatically improved the state of the art on a wide variety of tasks.
Pre-training strategies and datasets for facial representation learning
1adrianb/unsupervised-face-representation
•
•
Recent work on Deep Learning in the area of face analysis has focused on supervised learning for specific tasks of interest (e. g. face recognition, facial landmark localization etc.)
A Large-Scale Study on Unsupervised Spatiotemporal Representation Learning
facebookresearch/SlowFast
•
•
CVPR 2021
We present a large-scale study on unsupervised spatiotemporal representation learning from videos.
PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training
We also show that our method is able to utilize real-time human feedback to effectively prevent reward exploitation and learn new behaviors that are difficult to specify with standard reward functions.
Learning of feature points without additional supervision improves reinforcement learning from images
rinuboney/fpac
•
•
In many control problems that include vision, optimal controls can be inferred from the location of the objects in the scene.
RandomRooms: Unsupervised Pre-training from Synthetic Shapes and Randomized Layouts for 3D Object Detection
wyf-accept/backtoreality
•
•
ICCV 2021
In particular, we propose to generate random layouts of a scene by making use of the objects in the synthetic CAD dataset and learn the 3D scene representation by applying object-level contrastive learning on two random scenes generated from the same set of synthetic objects.
AI-Bind: Improving Binding Predictions for Novel Protein Targets and Ligands
Identifying novel drug-target interactions (DTI) is a critical and rate limiting step in drug discovery.
Unsupervised Pre-Training on Patient Population Graphs for Patient-Level Predictions
chantalmp/unsupervised-pre-training-on-patient-population-graphs-for-patient-level-predictions
•
•
We test our method on two medical datasets of patient records, TADPOLE and MIMIC-III, including imaging and non-imaging features and different prediction tasks.
Reinforcement Learning with Action-Free Pre-Training from Videos
younggyoseo/apv
•
•
Our framework consists of two phases: we pre-train an action-free latent video prediction model, and then utilize the pre-trained representations for efficiently learning action-conditional world models on unseen environments.
