Noncontact Multimodal Sensing for Driver Distraction Detection via Behavioral-Physiological Fusion

Distraction caused by external interference and fatigue is a major factor in traffic accidents. Due to the similarity in their external behavioral characteristics, conventional machine vision technologies struggle to effectively distinguish such behaviors, which pose challenges in formulating corresponding intervention strategies. This paper proposes a distracted driving detection algorithm based on key behavioral features fusion, which is evaluated on a self-constructed dataset. It can accurately identify driving behaviors induced by interference, such as head deviation, smoking, and phone use, achieving an overall recognition accuracy of 93.30%. Additionally, by integrating facial fatigue features and heart rate variability (HRV) features, a fatigue state recognition method is developed, with an accuracy of 94.25%. Unlike traditional contact-based sensors for heart rate measurement, this study uses remote photoplethysmography (rPPG) technology to remotely sense and capture HRV features related to fatigue through a camera. To enhance the robustness of the system under varying lighting conditions, a light-sensing mechanism is designed to adaptively switch between RGB and infrared cameras based on ambient illumination levels. This approach not only enables a single camera device to have multiple functions but also differentiates between interference-type and physiological-type distractions with an accuracy of 94.91%. The proposed method provides an effective technical solution for identifying abnormal behaviors, contributing to intelligent transportation and safer driving.