Orchestrate Latent Expertise: Advancing Online Continual Learning with Multi-Level Supervision and Reverse Self-Distillation

1 code implementation • 30 Mar 2024 • HongWei Yan, Liyuan Wang, Kaisheng Ma, Yi Zhong

However, a notable gap from CL to OCL stems from the additional overfitting-underfitting dilemma associated with the use of rehearsal buffers: the inadequate learning of new training samples (underfitting) and the repeated learning of a few old training samples (overfitting).

Continual Learning Knowledge Distillation