Browse SoTA > Methodology > Transfer Learning

Transfer Learning

546 papers with code · Methodology

Transfer learning is a methodology where weights from a model trained on one task are taken and either used (a) to construct a fixed feature extractor, (b) as weight initialization and/or fine-tuning.

( Image credit: Subodh Malgonde )

Leaderboards

Greatest papers with code

Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrieval

3 Apr 2020tensorflow/models

GLDv2 is the largest such dataset to date by a large margin, including over 5M images and 200k distinct instance labels.

IMAGE RETRIEVAL TRANSFER LEARNING

Talking-Heads Attention

5 Mar 2020tensorflow/models

We introduce "talking-heads attention" - a variation on multi-head attention which includes linearprojections across the attention-heads dimension, immediately before and after the softmax operation. While inserting only a small number of additional parameters and a moderate amount of additionalcomputation, talking-heads attention leads to better perplexities on masked language modeling tasks, aswell as better quality when transfer-learning to language comprehension and question answering tasks.

LANGUAGE MODELLING QUESTION ANSWERING TRANSFER LEARNING

Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data

18 Oct 2016tensorflow/models

The approach combines, in a black-box fashion, multiple models trained with disjoint datasets, such as records from different subsets of users.

TRANSFER LEARNING

HuggingFace's Transformers: State-of-the-art Natural Language Processing

9 Oct 2019huggingface/transformers

In this paper, we present HuggingFace's Transformers library, a library for state-of-the-art NLP, making these developments available to the community by gathering state-of-the-art general-purpose pretrained models under a unified API together with an ecosystem of libraries, examples, tutorials and scripts targeting many downstream NLP tasks.

TEXT GENERATION TRANSFER LEARNING

DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter

NeurIPS 2019 huggingface/transformers

As Transfer Learning from large-scale pre-trained models becomes more prevalent in Natural Language Processing (NLP), operating these large models in on-the-edge and/or under constrained computational training or inference budgets remains challenging.

LANGUAGE MODELLING LINGUISTIC ACCEPTABILITY NATURAL LANGUAGE INFERENCE QUESTION ANSWERING SEMANTIC TEXTUAL SIMILARITY SENTIMENT ANALYSIS TRANSFER LEARNING

Large-scale Simple Question Answering with Memory Networks

5 Jun 2015facebookresearch/ParlAI

Training large-scale question answering systems is complicated because training sources usually cover a small portion of the range of possible questions.

QUESTION ANSWERING TRANSFER LEARNING

Transfer Learning with Dynamic Distribution Adaptation

17 Sep 2019jindongwang/transferlearning

Since the source and the target domains are usually from different distributions, existing methods mainly focus on adapting the cross-domain marginal or conditional distributions.

DOMAIN ADAPTATION IMAGE CLASSIFICATION SENTIMENT ANALYSIS TRANSFER LEARNING

Learning What and Where to Transfer

15 May 2019jindongwang/transferlearning

To address the issue, we propose a novel transfer learning approach based on meta-learning that can automatically learn what knowledge to transfer from the source network to where in the target network.

META-LEARNING SMALL DATA IMAGE CLASSIFICATION TRANSFER LEARNING