A Quantitative Analysis of Catastrophic Forgetting in Quantized Spiking Neural Networks

Continual learning (CL) in artificial intelligence enables systems to adapt and accumulate knowledge over time. However, a significant challenge in CL is catastrophic forgetting, where models lose previously learned information when exposed to new data distributions. This study investigates the potential of spiking neural networks (SNNs) and their quantized variants in mitigating catastrophic forgetting. The rationale is that by using sparsely activated neurons, the weight updates are likewise sparse, which may slow the forgetting process. We conduct a comprehensive evaluation of both Quantized SNNs (QSNN) and Post-Quantized SNNs (PQSNNs) under varying quantization levels (8-bit, 4-bit, and 2-bit) across two distinct scenarios: digit recognition with shifting data distributions (MNIST and Permuted MNIST datasets) and complex temporal pattern learning (Spiking Heidelberg Digits dataset). This allows us to assess the models’ resilience under aggressive quantization regimes and diverse learning challenges. Our experimental framework rigorously tests the models for accuracy, loss, and activation sparsity to quantify the extent of catastrophic forgetting. Our findings indicate that PQSNNs 8-bit and 4-bit models are approximately on par with the baseline SNN, in terms of accuracy and test loss while minimizing significant compromise, side-stepping the need for retraining in the quantization-aware regime.