Dedicated Class Subnetworks for SNN Class Incremental Learning

We explore an unconventional approach to the Class Incremental Learning (CIL) problem that dedicates a separate subnetwork for the independent learning of each class. This separation ensures that the learning of a new class has no effect on the model’s previously acquired knowledge. We present a learning strategy loosely inspired by biological neuron apoptosis (neuron death) and neurogenesis (neuron birth) combined with other methods to achieve effective learning and minimize the model’s resource requirements. The network is entirely feed-forward and uses low precision inter-neuron spikes combined with simple neuron behaviors, making it suitable for very low power Spiking Neural Network (SNN) realization on appropriate future neuromorphic platforms. We demonstrate the model in an abstract setting to explore the tradeoffs between optimizing for accuracy, network size, and run-time costs and show that, despite no competition between the classes during learning, it is possible to achieve top-1 accuracy of 85% on MNIST.