Bayesian Meta-Learning for the Few-Shot Setting via Deep Kernels
Recently, different machine learning methods have been introduced to tackle the challenging few-shot learning scenario that is, learning from a small labeled dataset related to a specific task. Common approaches have taken the form of meta-learning: learning to learn on the new problem given the old. Following the recognition that meta-learning is implementing learning in a multi-level model, we present a Bayesian treatment for the meta-learning inner loop through the use of deep kernels. As a result we can learn a kernel that transfers to new tasks; we call this Deep Kernel Transfer (DKT). This approach has many advantages: is straightforward to implement as a single optimizer, provides uncertainty quantification, and does not require estimation of task-specific parameters. We empirically demonstrate that DKT outperforms several state-of-the-art algorithms in few-shot classification, and is the state of the art for cross-domain adaptation and regression. We conclude that complex meta-learning routines can be replaced by a simpler Bayesian model without loss of accuracy.
PDF Abstract NeurIPS 2020 PDF NeurIPS 2020 AbstractCode
Datasets
CUB-200-2011
EMNIST
Results from the Paper
| Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Benchmark |
|---|---|---|---|---|---|---|
| Few-Shot Image Classification | CUB 200 5-way 1-shot | DKT + BNCosSim | Accuracy | 72.27 | # 25 | |
| Few-Shot Image Classification | CUB 200 5-way 5-shot | DKT + BNCosSim | Accuracy | 85.64 | # 22 | |
| Few-Shot Image Classification | Mini-Imagenet 5-way (1-shot) | DKT + BNCosSim | Accuracy | 62.96 | # 58 | |
| Few-Shot Image Classification | Mini-Imagenet 5-way (5-shot) | DKT + BNCosSim | Accuracy | 64.0 | # 86 | |
| Few-Shot Image Classification | Mini-ImageNet-CUB 5-way (1-shot) | DKT + CosSim | Accuracy | 40.22 | # 8 | |
| Few-Shot Image Classification | Mini-ImageNet-CUB 5-way (5-shot) | DKT + BNCosSim | Accuracy | 56.40 | # 8 | |
| Few-Shot Image Classification | OMNIGLOT-EMNIST 5-way (1-shot) | DKT + BNCosSim | Accuracy | 75.40 | # 2 | |
| Few-Shot Image Classification | OMNIGLOT-EMNIST 5-way (5-shot) | DKT + BNCosSim | Accuracy | 90.3 | # 2 |
