Browse > Methodology > Transfer Learning

Transfer Learning

317 papers with code · Methodology

Transfer learning is a methodology where weights from a model trained on one task are taken and either used (a) to construct a fixed feature extractor, (b) as weight initialization and/or fine-tuning.

State-of-the-art leaderboards

Greatest papers with code

Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data

18 Oct 2016tensorflow/models

The approach combines, in a black-box fashion, multiple models trained with disjoint datasets, such as records from different subsets of users.

TRANSFER LEARNING

HuggingFace's Transformers: State-of-the-art Natural Language Processing

9 Oct 2019huggingface/transformers

In this paper, we present HuggingFace's Transformers library, a library for state-of-the-art NLP, making these developments available to the community by gathering state-of-the-art general-purpose pretrained models under a unified API together with an ecosystem of libraries, examples, tutorials and scripts targeting many downstream NLP tasks.

TEXT GENERATION TRANSFER LEARNING

DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter

2 Oct 2019huggingface/transformers

As Transfer Learning from large-scale pre-trained models becomes more prevalent in Natural Language Processing (NLP), operating these large models in on-the-edge and/or under constrained computational training or inference budgets remains challenging.

LANGUAGE MODELLING LINGUISTIC ACCEPTABILITY NATURAL LANGUAGE INFERENCE QUESTION ANSWERING SEMANTIC TEXTUAL SIMILARITY SENTIMENT ANALYSIS TRANSFER LEARNING

Large-scale Simple Question Answering with Memory Networks

5 Jun 2015facebookresearch/ParlAI

Training large-scale question answering systems is complicated because training sources usually cover a small portion of the range of possible questions.

QUESTION ANSWERING TRANSFER LEARNING

Learning What and Where to Transfer

15 May 2019jindongwang/transferlearning

To address the issue, we propose a novel transfer learning approach based on meta-learning that can automatically learn what knowledge to transfer from the source network to where in the target network.

META-LEARNING TRANSFER LEARNING

Easy Transfer Learning By Exploiting Intra-domain Structures

2 Apr 2019jindongwang/transferlearning

In this paper, we propose a practically Easy Transfer Learning (EasyTL) approach which requires no model selection and hyperparameter tuning, while achieving competitive performance.

MODEL SELECTION TRANSFER LEARNING

Bag of Tricks for Image Classification with Convolutional Neural Networks

CVPR 2019 dmlc/gluon-cv

Much of the recent progress made in image classification research can be credited to training procedure refinements, such as changes in data augmentations and optimization methods.

IMAGE CLASSIFICATION OBJECT DETECTION SEMANTIC SEGMENTATION TRANSFER LEARNING

DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition

6 Oct 2013jetpacapp/DeepBeliefSDK

We evaluate whether features extracted from the activation of a deep convolutional network trained in a fully supervised fashion on a large, fixed set of object recognition tasks can be re-purposed to novel generic tasks.

DOMAIN ADAPTATION OBJECT RECOGNITION SCENE RECOGNITION TRANSFER LEARNING

EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks

ICML 2019 lukemelas/EfficientNet-PyTorch

Convolutional Neural Networks (ConvNets) are commonly developed at a fixed resource budget, and then scaled up for better accuracy if more resources are available.

 SOTA for Image Classification on Stanford Cars (using extra training data)

FINE-GRAINED IMAGE CLASSIFICATION NEURAL ARCHITECTURE SEARCH TRANSFER LEARNING