Efficient Spiking Neural Networks Incorporating Temporal Attention Mechanisms and Surrogate Gradient

In recent years, Spiking Neural Networks (SNNs) have received much attention for their efficient bio-inspired properties, especially in processing time-series data. However, conventional SNNs face the challenges of information loss and insufficient temporal dependence in the encoding layer. To address these issues, this study proposes a SNN coding layer that integrates a temporal attention mechanism and introduces a new surrogate gradient function to improve the model's learning ability and performance. This study proposes a novel temporal attention mechanism designed to enhance the performance of the coding layer of SNNS. Traditional SNNs often face the problems of information loss and lack of temporal dependence when dealing with temporal data. To address these problems, we introduce a temporal attention mechanism, which enables the network to capture important temporal features more efficiently by dynamically adjusting the weights of the input information. In addition, we design a new gradient substitution function to accommodate the discrete nature of the impulse signal. This substitution function can improve the learning ability of the network by avoiding the common gradient vanishing problem in traditional backpropagation while maintaining the gradient flow. The experimental results show that the SNN combining the temporal attention mechanism and the new surrogate gradient function performs superiorly on several benchmark datasets, displaying stronger temporal information processing ability and higher classification accuracy.