Compacting, Picking and Growing for Unforgetting Continual Learning

Continual lifelong learning is essential to many applications. In this paper, we propose a simple but effective approach to continual deep learning. Our approach leverages the principles of deep model compression, critical weights selection, and progressive networks expansion. By enforcing their integration in an iterative manner, we introduce an incremental learning method that is scalable to the number of sequential tasks in a continual learning process. Our approach is easy to implement and owns several favorable characteristics. First, it can avoid forgetting (i.e., learn new tasks while remembering all previous tasks). Second, it allows model expansion but can maintain the model compactness when handling sequential tasks. Besides, through our compaction and selection/expansion mechanism, we show that the knowledge accumulated through learning previous tasks is helpful to build a better model for the new tasks compared to training the models independently with tasks. Experimental results show that our approach can incrementally learn a deep model tackling multiple tasks without forgetting, while the model compactness is maintained with the performance more satisfiable than individual task training.

PDF Abstract NeurIPS 2019 PDF NeurIPS 2019 Abstract
Task Dataset Model Metric Name Metric Value Global Rank Uses Extra
Training Data
Result Benchmark
Age And Gender Classification Adience Age CPG (single crop, pytorch) Accuracy (5-fold) 57.66 # 9
Age And Gender Classification Adience Gender CPG (single crop, pytorch) Accuracy (5-fold) 89.66 # 4
Facial Expression Recognition (FER) AffectNet CPG Accuracy (7 emotion) 63.57 # 18
Accuracy (8 emotion) - # 29
Continual Learning Cifar100 (20 tasks) CPG Average Accuracy 80.9 # 4
Continual Learning CUBS (Fine-grained 6 Tasks) CPG Accuracy 83.59 # 3
Continual Learning Flowers (Fine-grained 6 Tasks) CPG Accuracy 96.62 # 2
Continual Learning ImageNet (Fine-grained 6 Tasks) CPG Accuracy 75.81 # 4
Continual Learning Sketch (Fine-grained 6 Tasks) CPG Accuracy 80.33 # 2
Continual Learning Stanford Cars (Fine-grained 6 Tasks) CPG Accuracy 92.80 # 1
Continual Learning Wikiart (Fine-grained 6 Tasks) CPG Accuracy 77.15 # 2

Methods


No methods listed for this paper. Add relevant methods here