Low Latency Crossing Pedestrian Detection by Dynamic Vision Sensor and RGB camera

Pedestrian detection in autonomous driving systems is a critical task that demands high accuracy and low latency, especially in dynamic environments where rapid decision-making is essential. Traditional RGB-based detectors suffer from limitations such as motion blur, occlusions, and degraded performance in low-light conditions. To address these challenges, we propose a novel low-latency pedestrian detection approach that fuses Dynamic Vision Sensors (DVS) with traditional RGB cameras. We apply a geometry-based algorithm leveraging epipolar constraints to remove background events caused by ego-motion, thereby isolating dynamic events generated by pedestrians. We introduce a Likelihood Ratio Test (LRT)–based fusion framework that combines appearance-based confidence scores with the motion-sensitive Active Pixel Percentage (APP) derived from event data. This fusion approach effectively differentiates static and moving pedestrians, enhancing detection robustness. We evaluate the method on both synthetic data from the CARLA simulator and a real-recorded dataset. Across both domains, our fusion model significantly improves classification accuracy and achieves substantially earlier detection of pedestrians compared to RGB-only detection. These results highlight the potential of event-based sensor fusion for real-time pedestrian perception in autonomous driving and safety-critical robotics applications.