Transfer Learning
2841 papers with code • 7 benchmarks • 15 datasets
Transfer Learning is a machine learning technique where a model trained on one task is re-purposed and fine-tuned for a related, but different task. The idea behind transfer learning is to leverage the knowledge learned from a pre-trained model to solve a new, but related problem. This can be useful in situations where there is limited data available to train a new model from scratch, or when the new task is similar enough to the original task that the pre-trained model can be adapted to the new problem with only minor modifications.
( Image credit: Subodh Malgonde )
Benchmarks
These leaderboards are used to track progress in Transfer Learning
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
Libraries
Use these libraries to find Transfer Learning models and implementationsDatasets
KITTI
Office-Home
Sims4Action
MFRC
CARLANE Benchmark
BanglaLekha-Isolated
SpaRTUN
fluocells
Retinal Fundus MultiDisease Image Dataset (RFMiD)
Subtasks
Most implemented papers
Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles
By following the principles of self-supervision, we build a convolutional neural network (CNN) that can be trained to solve Jigsaw puzzles as a pretext task, which requires no manual labeling, and then later repurposed to solve object classification and detection.
Pruning Convolutional Neural Networks for Resource Efficient Inference
We propose a new criterion based on Taylor expansion that approximates the change in the cost function induced by pruning network parameters.
HuggingFace's Transformers: State-of-the-art Natural Language Processing
Transformer architectures have facilitated building higher-capacity models and pretraining has made it possible to effectively utilize this capacity for a wide variety of tasks.
Unveiling COVID-19 from Chest X-ray with deep learning: a hurdles race with small data
ieee8023/covid-chestxray-dataset
•
•
The possibility to use widespread and simple chest X-ray (CXR) imaging for early screening of COVID-19 patients is attracting much interest from both the clinical and the AI community.
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
We evaluate whether features extracted from the activation of a deep convolutional network trained in a fully supervised fashion on a large, fixed set of object recognition tasks can be re-purposed to novel generic tasks.
Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data
The approach combines, in a black-box fashion, multiple models trained with disjoint datasets, such as records from different subsets of users.
A Simple Baseline for Bayesian Uncertainty in Deep Learning
wjmaddox/swa_gaussian
•
•
NeurIPS 2019
We propose SWA-Gaussian (SWAG), a simple, scalable, and general purpose approach for uncertainty representation and calibration in deep learning.
GoEmotions: A Dataset of Fine-Grained Emotions
google-research/google-research
•
•
ACL 2020
Understanding emotion expressed in language has a wide range of applications, from building empathetic chatbots to detecting harmful online behavior.
Towards Compact Single Image Super-Resolution via Contrastive Self-distillation
Booooooooooo/CSD
•
•
Convolutional neural networks (CNNs) are highly successful for super-resolution (SR) but often require sophisticated architectures with heavy memory cost and computational overhead, significantly restricts their practical deployments on resource-limited devices.
Deep Hashing Network for Unsupervised Domain Adaptation
Domain adaptation or transfer learning algorithms address this challenge by leveraging labeled data in a different, but related source domain, to develop a model for the target domain.
