SPARK: Simple and Parameter-Free Knowledge Embedding With Fuzzy Cognitive Maps for Class Incremental Learning

Class incremental learning (CIL) aims to mitigate catastrophic forgetting of previously learned classes when integrating new knowledge. A primary challenge contributing to forgetting is the absence of data from earlier classes. Researchers have designed a variety of methods to solve the problem, among which topology-preserving methods show tremendous potential. However, two problems remain: (1) A large hyperparameter search space for constructing and utilizing a complex topology hinders efficient performance optimization; (2) Constraining the network to preserve the topology in the objective makes it difficult to optimize. This paper proposes SPARK, a simple and parameter-free method by embedding fuzzy cognitive maps, to address the problems. First, we construct a fuzzy cognitive map with nodes representing class prototypes and edges representing inter-class similarities. Then, we exploit the fuzzy cognitive map to obtain class-level embedding by aggregating features of other classes for each class. Finally, the class- and sample-level embeddings are fused and fed to the classifier. The proposed method can be easily optimized without introducing additional loss terms and hyperparameters. We theoretically prove that such a simple fuzzy cognitive map embedding can efficiently preserve the structural information of the fuzzy cognitive map. Experimental results indicate that SPARK achieves up to 5.85% higher average accuracy and 8.69% reduction in forgetting compared to baseline methods. The code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/helloxjb/SPARK/</uri>