LiS-Net: A Brain-Inspired Framework for Event-Based End-to-End Steering Prediction

The advancement of autonomous vehicles has shifted from modular pipeline architectures to end-to-end frameworks, enabling direct learning of control policies from sensory inputs. While frame-based RGB cameras are commonly utilized, they face challenges in dynamic environments, such as motion blur and varying illumination. Alternatively, event-based cameras, with their high temporal resolution and wide dynamic range, offer a promising solution. However, existing end-to-end models for event camera inputs are primarily constructed using traditional convolutional networks and time-sequence models (e.g., Recurrent Neural Networks, RNNs), which suffer from large parameter counts and excessive redundant computations. To address this gap, we propose LiS-Net, a novel framework that incorporates brain-inspired neural networks to construct the overall architecture, applying it to the task of end-to-end steering prediction. The core of LiS-Net is a liquid neural network, which is designed to simulate the behavior of C. elegans neurons for modeling purposes. By leveraging the strengths of event cameras and brain-inspired computation, LiS-Net achieves superior accuracy, smoothness, and efficiency. Specifically, LiS-Net outperforms existing models with the lowest RMSE and MAE, indicating better accuracy, while also maintaining the fewest number of neurons and achieving competitive FLOPs results, showcasing its computational efficiency. Experiments on the simulated EventScape dataset demonstrate its robustness, while validation on our self-collected dataset showcases its generalization capability. We also release the collected dataset comprising synchronized event cameras, RGB cameras, and GPS and CAN data. LiS-Net lays the foundation for scalable and efficient autonomous driving solutions by integrating bio-inspired sensors with brain-inspired computation.