Distributed Heterogeneous Spiking Neural Network Simulator Using Sunway Accelerators

Spiking Neural Network (SNN) simulation is very important for studying brain function and validating the hypotheses for neuroscience, and it can also be used in artificial intelligence. Recently, GPU-based simulators have been developed to support the real-time simulation of SNN. However, these simulators' simulating performance and scale are severely limited, due to the random memory access pattern and the global communication between devices. Therefore, we propose an efficient distributed heterogeneous SNN simulator based on the Sunway accelerators (including SW26010 and SW26010pro), named SWsnn, which supports accurate simulation with small time step (1/16 ms), randomly delay sizes for synapses, and larger scale network computing. Compared with existing GPUs, the Local Dynamic Memory (LDM) (similar to cache) in Sunway is much bigger (4 MB or 16 MB in each core group). To improve the simulation performance, we redesign the network data storage structure and the synaptic plasticity flow to make most random accesses occur in LDM. SWsnn hides Message Passing Interface (MPI)-related operations to reduce communication costs by separating SNN general workflow. Besides, SWsnn relies on parallel Compute Processing Elements (CPEs) rather than serial Manage Processing Element (MPE) to control the communicating buffers, using Register-Level Communication (RLC) and Direct Memory Access (DMA). In addition, SWsnn is further optimized using vectorization and DMA hiding techniques. Experimental results show that SWsnn runs 1.4-2.2 times faster than state-of-the-art GPU-based SNN simulator GPU-enhanced Neuronal Networks (GeNN), and supports much larger scale real-time simulation.