Search Results for author:
Found 7 papers, 7 papers with code
IMEX-Reg: Implicit-Explicit Regularization in the Function Space for Continual Learning
To further leverage the global relationship between representations learned using CRL, we propose a regularization strategy to guide the classifier toward the activation correlations in the unit hypersphere of the CRL.
TriRE: A Multi-Mechanism Learning Paradigm for Continual Knowledge Retention and Promotion
Continual learning (CL) has remained a persistent challenge for deep neural networks due to catastrophic forgetting (CF) of previously learned tasks.
BiRT: Bio-inspired Replay in Vision Transformers for Continual Learning
The ability of deep neural networks to continually learn and adapt to a sequence of tasks has remained challenging due to catastrophic forgetting of previously learned tasks.
Task-Aware Information Routing from Common Representation Space in Lifelong Learning
Thus, inspired by the Global Workspace Theory of conscious information access in the brain, we propose TAMiL, a continual learning method that entails task-attention modules to capture task-specific information from the common representation space.
Task Agnostic Representation Consolidation: a Self-supervised based Continual Learning Approach
Furthermore, the domain shift between pre-training data distribution and the task distribution reduces the generalizability of the learned representations.
Consistency is the key to further mitigating catastrophic forgetting in continual learning
Therefore, we examine the role of consistency regularization in ER framework under various continual learning scenarios.
Distill on the Go: Online knowledge distillation in self-supervised learning
To address the issue of self-supervised pre-training of smaller models, we propose Distill-on-the-Go (DoGo), a self-supervised learning paradigm using single-stage online knowledge distillation to improve the representation quality of the smaller models.
