Advanced Pedestrian Detection in Low Visibility Scenarios Using UltraLytics YOLOv8 and Kalman Filtering

Nighttime detection of pedestrians brings important problems such as low light conditions, limitations of sensors, and environmental noise. To tackle this problem, in this study, we propose a novel visual-radar data fusion to enhance a pedestrian detection system using the UltraLytics YOLOv8 deep learning model together with Kalman Filtering technique. The system is based on a Low-Light Vision Infrared Paired (LLVIP) dataset that includes infrared vision data to attenuate the performance decrement of a pedestrian detector in the dark. The combination of infrared cameras and millimeter-wave radar achieves a detection rate of 97.6<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">%</sup>, in which UltraLytics YOLOv8 exhibits excellent object recognition performance by generating accurate spatial & category information from the infrared image; complementary distance and velocity data are obtained by millimeter-wave radar. It brings Kalman Filtering to track the radar-detected pedestrian targets, making the system more robust by predicting target and noise reduction. It was capable of fusing infrared and radar data at decision-level to make the system provide very precise position accurately in even nighttime situations. The experimental results show that the proposed model has improved capability of detection accuracy, robustness, and response time in comparison with the single-sensor system, which is suitable for practical autonomous driving scenarios.