Low-latency neuromorphic air hockey player

Abstract Brains process sensory information to guide behaviour, enabling organisms to adapt to dynamic and unpredictable conditions. Neuromorphic engineering seeks to emulate these neurobiological principles to develop compact, low-power systems capable of real-time sensory-motor integration. This approach addresses some limitations of traditional AI and holds promise for autonomous systems that can interact robustly with the real world. However, most of today’s widely used neuromorphic benchmarks focus primarily on improving accuracy metrics using pre-recorded datasets, often overlooking critical factors such as latency and power consumption. This underscores the need for benchmarks to evaluate real-time performance under noisy, dynamic conditions. To address this need, we developed a system that uses spiking neural networks (SNNs) to control a robotic manipulator in an air-hockey game. In this setup, the automated opponent uses SNNs to process data from an event-based camera, enabling it to track the puck’s movements and respond to the actions of a human player. Our study demonstrates the potential of SNNs to accomplish fast real-time tasks while running on massively parallel hardware. We believe our air-hockey platform provides a versatile testbed for evaluating neuromorphic systems and invites further exploration of advanced algorithms, such as those incorporating trajectory prediction or adaptive learning, which could significantly enhance real-time decision-making and control.