Path Masking Technique for Efficient Continual Learning in Task Representation Space

Since neural networks learn using global weights, sequential training of different datasets on the same neural network leads to the phenomenon of catastrophic forgetting, where the weights for previously trained datasets change and the existing knowledge is forgotten. To solve this problem, various Continuous Learning methods such as Elastic Weight Consolidation (EWC) and Replay techniques have been proposed, but most of them require continuity between similar domains. In this study, we propose a path masking technique that can mitigate the forgetting phenomenon even if datasets with different domains are trained sequentially. Path Masking partitions the activated neuronal pathways according to the dataset to induce a natural separation of representations while sharing parameters. Unlike the previously proposed PathNet, Piggyback, and SupSup, it is differentiated in that it can be easily applied to the existing underlying neural network structure without path exploration or additional weight fixation. Experimental results show that the proposed method can effectively mitigate the forgetting phenomenon through representation separation in Deep Neural Network (DNN) and Convolutional Neural Network (CNN) models, and can simultaneously represent different domains with one neural network.