Task Separation and Knowledge Sharing for Class Incremental Learning

Methods based on dynamically expanding architectures can effectively mitigate catastrophic forgetting in class incremental learning (CIL), but they often overlook information sharing and integration between subnetworks, leading to performance degradation and increased parameter count and memory consumption as the network expands. To address these complex issues, we introduce a new method called Task Separation and Knowledge Sharing (TSKS), which transfers knowledge at both the logical and feature levels through Hybrid Knowledge Distillation (HKD). In logical distillation, we combine Global Knowledge Distillation (GKD) and Task Knowledge Distillation (TKD) to establish connections between tasks. To prevent the model from favoring new classes, we design an Attention- Based Feature Fusion (ABFF) mechanism to capture important features. Additionally, we introduce a Soft-Margin Auxiliary Loss (SMAL) to help the model learn new concepts. Extensive experiments show that TSKS outperforms existing models on CIFAR100 and ImageNet100, regardless of whether memory samples are used. Notably, TSKS demonstrates superior performance in memory-free settings, indicating a reduced reliance on rehearsal memory.