Unsupervised Pre-training
104 papers with code • 2 benchmarks • 7 datasets
Pre-training a neural network using unsupervised (self-supervised) auxiliary tasks on unlabeled data.
Benchmarks
These leaderboards are used to track progress in Unsupervised Pre-training
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
Libraries
Use these libraries to find Unsupervised Pre-training models and implementations
CARLANE Benchmark
m2caiSeg
GBUSV
TYC Dataset
Icon645
Latest papers
Pre-training Contextualized World Models with In-the-wild Videos for Reinforcement Learning
thuml/ContextWM
•
•
NeurIPS 2023
To tackle this issue, we introduce Contextualized World Models (ContextWM) that explicitly separate context and dynamics modeling to overcome the complexity and diversity of in-the-wild videos and facilitate knowledge transfer between distinct scenes.
Rethinking Semi-supervised Learning with Language Models
zhengxiangshi/powerfulpromptft
•
•
Semi-supervised learning (SSL) is a popular setting aiming to effectively utilize unlabelled data to improve model performance in downstream natural language processing (NLP) tasks.
PTGB: Pre-Train Graph Neural Networks for Brain Network Analysis
owen-yang-18/brainnn-pretrain
•
•
The human brain is the central hub of the neurobiological system, controlling behavior and cognition in complex ways.
LATTE: Label-efficient Incident Phenotyping from Longitudinal Electronic Health Records
celehs/latte
•
•
We propose a LAbel-efficienT incidenT phEnotyping (LATTE) algorithm to accurately annotate the timing of clinical events from longitudinal EHR data.
Don't Stop Pretraining? Make Prompt-based Fine-tuning Powerful Learner
Language models (LMs) trained on vast quantities of unlabelled data have greatly advanced the field of natural language processing (NLP).
PUNR: Pre-training with User Behavior Modeling for News Recommendation
ma787639046/punr
•
•
Firstly, we introduce the user behavior masking pre-training task to recover the masked user behaviors based on their contextual behaviors.
Unsupervised Pre-Training For Data-Efficient Text-to-Speech On Low Resource Languages
cnaigithub/SpeechDewarping
•
•
We empirically demonstrate the effectiveness of our proposed method in low-resource language scenarios, achieving outstanding performance compared to competing methods.
MultiTalent: A Multi-Dataset Approach to Medical Image Segmentation
Our findings offer a new direction for the medical imaging community to effectively utilize the wealth of available data for improved segmentation performance.
Generalized 3D Self-supervised Learning Framework via Prompted Foreground-Aware Feature Contrast
KangchengLiu/FAC_Foreground_Aware_Contrast
•
•
CVPR 2023
The second is that we prevent over-discrimination between 3D segments/objects and encourage grouped foreground-to-background distinctions at the segment level with adaptive feature learning in a Siamese correspondence network, which adaptively learns feature correlations within and across point cloud views effectively.
DocILE Benchmark for Document Information Localization and Extraction
rossumai/docile
•
•
This paper introduces the DocILE benchmark with the largest dataset of business documents for the tasks of Key Information Localization and Extraction and Line Item Recognition.
