Reducing spiking neural network inference latency for behavior cloning with input adaptive leakage

Abstract Behavior cloning enables efficient imitation of complex human skills from a limited number of demonstrations. However, artificial neural networks (ANNs), which are commonly employed for policy learning, suffer from their high computational demands and energy consumption. Spiking neural networks (SNNs) offer a promising alternative because of their event-driven and energy-efficient characteristics. A common training approach for SNNs is to convert the weights from a pretrained ANN into an equivalent SNN. However, due to rate coding, such a converted SNN typically requires multiple inference timesteps for each input to converge firing rates, resulting in increased latency. This issue is especially problematic for continuous sensory inputs such as video frames in robot manipulation. To address this, we propose an input-adaptive leakage mechanism that dynamically controls the leakage of the membrane potential, an internal state of spiking neurons, according to input variations. When the change between successive inputs is small, the mechanism decreases the leakage to preserve the membrane potential. Conversely, when the input change is large, it increases the leakage to rapidly update the membrane potential with the new input. We evaluate the proposed method using a hybrid ANN–SNN policy in which only the visual encoder is converted into an SNN. Experiments are conducted on four robotic manipulation tasks in the RoboManipBaselines simulation environment and one real-robot manipulation task. Compared with conventional reset-based SNNs, the proposed method reduces inference timesteps per frame by up to 87.5% and the measured energy of the entire hybrid policy per episode by 22.6% on average. These benefits diminish when inter-frame variations are large and broadly distributed, in which case the proposed method may require more timesteps and energy. Overall, the proposed method improves the efficiency of conventional SNN-based policies, although ANNs remain more efficient on the evaluated CPU and GPU platforms.