Analysis and Optimization of Spiking Neural Network Simulations on GPUs

Artificial neural networks (ANNs) have experienced remarkable growth over the past 20 years, propelled by advancements in GPU devices and the creation of accessible tools for constructing complex models. Concurrently, another type of neural network, spiking neural networks (SNNs), has been under development, particularly within neuroscience. SNNs, which are inspired by the brain, mimic biological nervous systems and incorporate neuronal dynamics described by differential equations. Recently, SNNs have gained traction in machine learning due to their potential for energy efficiency and relevance to discussions on artificial general intelligence (AGI). Over the past few decades, substantial research has been conducted on SNN simulation, leading to significant performance improvements, primarily through GPU-based implementations. However, there is a lack of benchmark studies involving the most recent GPU-based SNN simulators. In this work, we systematically analyze the state-of-the-art GPU-based SNN simulations and develop a novel simulator, cuSpike, which outperforms existing simulators on selected benchmark models by preserving accurate simulation results and providing energy-efficient executions.