Meta-SGD: Learning to Learn Quickly for Few-Shot Learning

31 Jul 2017  ·  Zhenguo Li, Fengwei Zhou, Fei Chen, Hang Li ·

Few-shot learning is challenging for learning algorithms that learn each task in isolation and from scratch. In contrast, meta-learning learns from many related tasks a meta-learner that can learn a new task more accurately and faster with fewer examples, where the choice of meta-learners is crucial. In this paper, we develop Meta-SGD, an SGD-like, easily trainable meta-learner that can initialize and adapt any differentiable learner in just one step, on both supervised learning and reinforcement learning. Compared to the popular meta-learner LSTM, Meta-SGD is conceptually simpler, easier to implement, and can be learned more efficiently. Compared to the latest meta-learner MAML, Meta-SGD has a much higher capacity by learning to learn not just the learner initialization, but also the learner update direction and learning rate, all in a single meta-learning process. Meta-SGD shows highly competitive performance for few-shot learning on regression, classification, and reinforcement learning.

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Few-Shot Image Classification Mini-Imagenet 20-way (1-shot) Meta SGD Accuracy 17.56 # 3
Few-Shot Image Classification Mini-Imagenet 20-way (1-shot) MAML, (from ) Accuracy 16.49 # 6
Few-Shot Image Classification Mini-Imagenet 20-way (1-shot) Meta LSTM, (from ) Accuracy 16.70 # 5
Few-Shot Image Classification Mini-Imagenet 20-way (1-shot) Matching Nets, (from ) Accuracy 17.31 # 4
Few-Shot Image Classification Mini-Imagenet 20-way (5-shot) MAML, (from ) Accuracy 19.29 # 6
Few-Shot Image Classification Mini-Imagenet 20-way (5-shot) Matching Nets, (from ) Accuracy 22.69 # 5
Few-Shot Image Classification Mini-Imagenet 20-way (5-shot) Meta LSTM, (from ) Accuracy 26.06 # 4
Few-Shot Image Classification Mini-Imagenet 20-way (5-shot) Meta SGD Accuracy 28.92 # 3

Methods