A Closer Look at Few-shot Classification

Few-shot classification aims to learn a classifier to recognize unseen classes during training with limited labeled examples. While significant progress has been made, the growing complexity of network designs, meta-learning algorithms, and differences in implementation details make a fair comparison difficult. In this paper, we present 1) a consistent comparative analysis of several representative few-shot classification algorithms, with results showing that deeper backbones significantly reduce the performance differences among methods on datasets with limited domain differences, 2) a modified baseline method that surprisingly achieves competitive performance when compared with the state-of-the-art on both the \miniI and the CUB datasets, and 3) a new experimental setting for evaluating the cross-domain generalization ability for few-shot classification algorithms. Our results reveal that reducing intra-class variation is an important factor when the feature backbone is shallow, but not as critical when using deeper backbones. In a realistic cross-domain evaluation setting, we show that a baseline method with a standard fine-tuning practice compares favorably against other state-of-the-art few-shot learning algorithms.

PDF Abstract ICLR 2019 PDF ICLR 2019 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Few-Shot Image Classification Dirichlet CUB-200 (5-way, 1-shot) Baseline++ 1:1 Accuracy 69.4 # 6
Few-Shot Image Classification Dirichlet CUB-200 (5-way, 5-shot) Baseline++ 1:1 Accuracy 87.5 # 4
Few-Shot Image Classification Dirichlet Mini-Imagenet (5-way, 1-shot) Baseline ++ 1:1 Accuracy 60.4 # 7
Few-Shot Image Classification Dirichlet Mini-Imagenet (5-way, 5-shot) Baseline++ 1:1 Accuracy 79.7 # 6
Few-Shot Image Classification Dirichlet Tiered-Imagenet (5-way, 1-shot) Baseline++ 1:1 Accuracy 68.0 # 7
Few-Shot Image Classification Dirichlet Tiered-Imagenet (5-way, 5-shot) Baseline++ 1:1 Accuracy 84.2 # 7
Few-Shot Image Classification Mini-ImageNet-CUB 5-way (1-shot) Baseline++ (Chen et al., 2019) Accuracy 33.04 # 12
Few-Shot Image Classification Mini-ImageNet-CUB 5-way (5-shot) Baseline++ (Chen et al., 2019) Accuracy 62.04 # 6

Methods


No methods listed for this paper. Add relevant methods here