A Dynamic Vision Sensor Object Recognition Model with Fusion of Multi-spiking Attention Mechanisms

Spiking Neural Networks (SNNs) utilize sparse spikes to transmit and process information, offering advantages such as low power consumption and reduced latency. This architecture is naturally well-suited for handling event-stream data generated by Dynamic Vision Sensors (DVSs). Attention mechanisms have been used to extract features of dynamic visual sensor objects by SNNs, which are inspired by the human optic nervous system. They give greater weight to parts of the input that contain focused information to extract relevant features. However, they often fail to consider some spatial and temporal domain information, including the edge features inherent of DVS objects and the temporal features of individual events. To improve the current attention mechanism feature extraction capability of SNNs, we propose a multi-spiking attention mechanisms fusion model. In order to fully use the temporal information of event streams, before attention mechanisms the two branches use proposed spatio-temporal feature statistics layer based on neuronal membrane potential leakage mechanism and proposed trainable event-driven convolution with trainable convolution kernel to extract primary features from raw event streams, the spiking spatio-temporal attention mechanism and spiking self-attention mechanism is used to further extract dependence features in each branch respectively. Finally feature fusion module to fuse the above features, which helps the model to extract the multidimensional features of DVS objects, which helps the model to extract the multidimensional features of DVS objects. Our model achieves classification performance on two neuromorphic datasets that outperform current SOTA or competitively. The visual analyses and other experiments further demonstrate the superior feature extraction capability of our model. It has been shown that our model can improve the performance of current attention mechanism SNNs for DVS objects.