Rethinking spikes in spiking neural networks for performance enhancement

The asynchronous and sparse characteristics of spiking neural networks (SNNs) have posited a promising potential to at least overcome the power consumption challenge encountered by today’s powerful artificial neural networks (ANNs). Neurons in SNNs communicate with each other and learn tasks through spikes, the activity of which represents the underlying neuronal state, i.e. the membrane potential, that integrates afferent inputs. A typically neuron used single threshold (ST) can only sample and represent information of a limited part in the rich dynamics of neuronal membrane potential. In addition, the width of nonzero area of surrogate gradient for ST neurons is restricted. Therefore, we argue that this commonly used single threshold (ST) will limit the expressiveness of spiking neurons and thus their performance. In this paper, we present a double-threshold (DT) scheme that extends the neuron’s capability to represent underlying information with both positive and negative precise spikes and prevent the neurons to be dormant. We show that our DT scheme can in principle enhance information transmission in SNNs. Our method can be easily generalized to various spike-based learning algorithms with a consistent improvement in performance. Our results show competitive accuracy on all examined datasets, highlighting the significance of our scheme to further improve the performance of SNNs.