Prototype Alignment with LoRA Fusion for Class-Incremental Learning

Recent advancements in pre-trained models have enhanced performance on downstream tasks due to their strong generalizability. Despite this, models fine-tuned continually often face challenges such as catastrophic forgetting and loss of generalization. To address these issues, we propose a novel approach that utilizes distinct Low-Rank Adaptation (LoRA) modules for each task. These modules parameter-efficient, and integrated across tasks to ensure the model maintains strong performance on both old and new classes. Additionally, we investigate semantic relationships between class prototypes to effectively reconstruct old prototypes in the context of new tasks. Our experiments demonstrate that this method significantly outperforms baseline approaches across various class-incremental learning benchmarks, offering an efficient and effective solution for mitigating forgetting and preserving model performance.