Continual Learning

799 papers with code • 28 benchmarks • 30 datasets

Continual Learning (also known as Incremental Learning, Life-long Learning) is a concept to learn a model for a large number of tasks sequentially without forgetting knowledge obtained from the preceding tasks, where the data in the old tasks are not available anymore during training new ones.
If not mentioned, the benchmarks here are Task-CL, where task-id is provided on validation.

Source:
Continual Learning by Asymmetric Loss Approximation with Single-Side Overestimation
Three scenarios for continual learning
Lifelong Machine Learning
Continual lifelong learning with neural networks: A review

Libraries

Use these libraries to find Continual Learning models and implementations
23 papers
1,644
6 papers
669
6 papers
450
See all 7 libraries.

Online Continual Learning For Interactive Instruction Following Agents

snumprlab/cl-alfred 12 Mar 2024

To take a step towards a more realistic embodied agent learning scenario, we propose two continual learning setups for embodied agents; learning new behaviors (Behavior Incremental Learning, Behavior-IL) and new environments (Environment Incremental Learning, Environment-IL) For the tasks, previous 'data prior' based continual learning methods maintain logits for the past tasks.

5
12 Mar 2024

Continual All-in-One Adverse Weather Removal with Knowledge Replay on a Unified Network Structure

xiaojihh/cl_all-in-one 12 Mar 2024

It considers the characteristics of the image restoration task with multiple degenerations in continual learning, and the knowledge for different degenerations can be shared and accumulated in the unified network structure.

1
12 Mar 2024

Federated Learning of Socially Appropriate Agent Behaviours in Simulated Home Environments

nchuramani/fcl-mannersdb 12 Mar 2024

In this paper, we present a novel FL benchmark that evaluates different strategies, using multi-label regression objectives, where each client individually learns to predict the social appropriateness of different robot actions while also sharing their learning with others.

1
12 Mar 2024

Premonition: Using Generative Models to Preempt Future Data Changes in Continual Learning

cl-premonition/premonition 12 Mar 2024

We show here that the combination of a large language model and an image generation model can similarly provide useful premonitions as to how a continual learning challenge might develop over time.

0
12 Mar 2024

On the Diminishing Returns of Width for Continual Learning

vihan-lakshman/diminishing-returns-wide-continual-learning 11 Mar 2024

While deep neural networks have demonstrated groundbreaking performance in various settings, these models often suffer from \emph{catastrophic forgetting} when trained on new tasks in sequence.

0
11 Mar 2024

LLMs in the Imaginarium: Tool Learning through Simulated Trial and Error

microsoft/simulated-trial-and-error 7 Mar 2024

We find that existing LLMs, including GPT-4 and open-source LLMs specifically fine-tuned for tool use, only reach a correctness rate in the range of 30% to 60%, far from reliable use in practice.

79
07 Mar 2024

Contrastive Continual Learning with Importance Sampling and Prototype-Instance Relation Distillation

lijy373/cclis 7 Mar 2024

Recently, because of the high-quality representations of contrastive learning methods, rehearsal-based contrastive continual learning has been proposed to explore how to continually learn transferable representation embeddings to avoid the catastrophic forgetting issue in traditional continual settings.

1
07 Mar 2024

GUIDE: Guidance-based Incremental Learning with Diffusion Models

cywinski/guide 6 Mar 2024

We introduce GUIDE, a novel continual learning approach that directs diffusion models to rehearse samples at risk of being forgotten.

3
06 Mar 2024

Interactive Continual Learning: Fast and Slow Thinking

biqing-qi/interactive-continual-learning-fast-and-slow-thinking 5 Mar 2024

Drawing on Complementary Learning System theory, this paper presents a novel Interactive Continual Learning (ICL) framework, enabled by collaborative interactions among models of various sizes.

52
05 Mar 2024

Recall-Oriented Continual Learning with Generative Adversarial Meta-Model

bigdata-inha/recall-oriented-cl-framework 5 Mar 2024

The stability-plasticity dilemma is a major challenge in continual learning, as it involves balancing the conflicting objectives of maintaining performance on previous tasks while learning new tasks.

4
05 Mar 2024