In-Progress: Enhancing Traffic Signal Perception for Connected and Autonomous Vehicles (CAVs) via Multi-Sensor Fusion of Camera, LiDAR, Radar, and SPaT Data

The ability to perceive a traffic signal is one of the most critical challenges in Connected and Autonomous Vehicles under adverse weather or sensor occlusions. Cameras, being the leading sensors for detecting traffic lights, have degraded performances under low visibility, glare, or physical obstruction. This work presents a multi-sensor fusion framework that combines data from cameras, LiDAR, Radar, and Signal Phase and Timing (SPaT) data from V2X communication in order to perceive traffic signals more accurately. It utilizes LiDAR for 3D spatial information, Radar for contextual information, and SPaT for ground-truth validation. Deep learning-based fusion model weights sensor inputs dynamically. We anticipate that this approach would significantly improve the accuracy, robustness against adverse conditions, and efficiency of decision-making compared to single or dual sensor approaches.