Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks

ICML 2017  ·  Chelsea Finn, Pieter Abbeel, Sergey Levine ·

We propose an algorithm for meta-learning that is model-agnostic, in the sense that it is compatible with any model trained with gradient descent and applicable to a variety of different learning problems, including classification, regression, and reinforcement learning. The goal of meta-learning is to train a model on a variety of learning tasks, such that it can solve new learning tasks using only a small number of training samples. In our approach, the parameters of the model are explicitly trained such that a small number of gradient steps with a small amount of training data from a new task will produce good generalization performance on that task. In effect, our method trains the model to be easy to fine-tune. We demonstrate that this approach leads to state-of-the-art performance on two few-shot image classification benchmarks, produces good results on few-shot regression, and accelerates fine-tuning for policy gradient reinforcement learning with neural network policies.

PDF Abstract ICML 2017 PDF ICML 2017 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Benchmark
Few-Shot Image Classification Dirichlet Mini-Imagenet (5-way, 1-shot) MAML 1:1 Accuracy 47.6 # 12
Few-Shot Image Classification Dirichlet Mini-Imagenet (5-way, 5-shot) MAML 1:1 Accuracy 64.5 # 11
Few-Shot Image Classification Meta-Dataset fo-MAML Accuracy 57.024 # 17
Few-Shot Image Classification Meta-Dataset Rank fo-MAML Mean Rank 10.25 # 10
Few-Shot Image Classification Mini-Imagenet 10-way (1-shot) MAML Accuracy 31.3 # 13
Few-Shot Image Classification Mini-Imagenet 10-way (1-shot) MAML + Transduction Accuracy 31.8 # 12
Few-Shot Image Classification Mini-Imagenet 10-way (5-shot) MAML Accuracy 46.9 # 13
Few-Shot Image Classification Mini-Imagenet 10-way (5-shot) MAML + Transduction Accuracy 48.2 # 10
Few-Shot Image Classification OMNIGLOT - 1-Shot, 5-way MAML Accuracy 98.7 # 10
Few-Shot Image Classification OMNIGLOT - 5-Shot, 5-way MAML Accuracy 99.9 # 2
Few-Shot Image Classification Tiered ImageNet 10-way (1-shot) MAML Accuracy 34.4 # 12
Few-Shot Image Classification Tiered ImageNet 10-way (1-shot) MAML + Transduction Accuracy 34.8 # 11
Few-Shot Image Classification Tiered ImageNet 10-way (5-shot) MAML Accuracy 53.3 # 11
Few-Shot Image Classification Tiered ImageNet 10-way (5-shot) MAML + Transduction Accuracy 54.7 # 10

Results from Other Papers


Task Dataset Model Metric Name Metric Value Rank Source Paper Compare
Few-Shot Image Classification Mini-Imagenet 5-way (1-shot) MAML Accuracy 48.7 # 96
Few-Shot Image Classification Mini-Imagenet 5-way (5-shot) MAML Accuracy 63.1 # 87
Few-Shot Image Classification Mini-ImageNet-CUB 5-way (1-shot) MAML (Finn et al., 2017) Accuracy 40.15 # 9
Image Classification Tiered ImageNet 5-way (5-shot) MAML Accuracy 70.30 # 5
Image Classification Tiered ImageNet 5-way (5-shot) MAML+Transduction Accuracy 70.83 # 4

Methods