Multi-Layer Distillation and Prototype Replay for Class-Incremental Learning

The primary challenge in class-incremental learning is mitigating catastrophic forgetting, where a model’s performance on previously learned tasks deteriorates drastically as it adapts to new tasks. This problem becomes even more pronounced in real-world applications, where the new data is constantly emerging, and retraining from scratch is not feasible. In this paper, we propose a class-incremental learning method called MDPR, which integrates several techniques, including self-supervised data augmentation, multi-layer knowledge distillation, and enhanced prototype replay. These strategies collectively improve the feature diversity extracted by the model, preserve knowledge of previous tasks, and address the issue of class imbalance in the classifier. Experimental results on the CIFAR-100 benchmarks demonstrate that MDPR effectively mitigates catastrophic forgetting, achieving high performance in class-incremental learning tasks.