Hybrid Neural Networks: Integrating Spikes for Energy Efficient Computation

In the quest for energy-efficient neural network architectures, Spiking Neural Networks (SNNs) have emerged as a promising alternative to traditional Artificial Neural Networks (ANNs). However, the inherent SNN training complexity and the high spiking activity often limit their practical application. This paper explores hybrid neural networks (HNNs) that integrate ANN and SNN layers to optimize the trade-off between performance and energy efficiency. Our study systematically explores nearly exhaustive configurations of ANNs and SNNs on two datasets (CIFAR-10 and GSC V2) and five architectures (VGG16, VGG11, ResNet18, ResNet8, and a 4-layer CNN). Depending on the proportion of SNN and ANN layers, our hybrid models can reduce up to 2.45 times the energy consumption while maintaining the same ANN baseline accuracy, and even exceeding it for some models. These results highlight the potential of hybrid models in the development of energy-efficient neural networks and paves the way for future hardware implementations.