Research on a Safe Driving Monitoring System Based on Multimodal Data
Abstract To address the safety issues in construction machinery caused by abnormal driving behaviors, this paper proposes a Multimodal Safe Driving Monitoring System (MSDMS). The system consists of two main components. The first is the Multimodal Sentiment Analysis model based on Hybrid Attention (MSA-HA), designed for emotional driving detection. By integrating hybrid attention mechanisms with a Tensor Fusion Network, this model enhances multimodal feature interaction and improves the accuracy of emotion detection. The second component employs the YOLOv8n-DPCLN model for the detection of fatigue and distracted driving. In this model, a Deepwise Partial Convolution Layer Network (DPCLN) replaces the bottleneck within the C2f module to enhance overall performance. Experimental results demonstrate that the MSA-HA achieves binary and ternary classification accuracies of 83.5% and 74.76%, respectively, on the SIMSv2 dataset, outperforming other mainstream models. Compared to YOLOv8n, the YOLOv8n-DPCLN improves [email protected] and [email protected]–0.95 by 1.9% and 3.7%, respectively, while reducing the parameter count by 0.4 M and computational cost by 1.1 GFLOPs.Finally, MSDMS achieves recognition rates of 78.7%, 93%, and 86% for emotion, fatigue, and distracted driving, respectively, thereby satisfying the requirements for safe driving monitoring.
Authors
- Lingling Tian (ORCID: https://orcid.org/0000-0001-5067-1572)
- Zhang Dengyin
- hao Cui
- Zepeng Wei
- Jie Cai
Institutions
- Nanjing University of Posts and Telecommunications (CN)
Publication Details
- Journal
- Research Square
- Published
- 2026-11-17
- DOI
- https://doi.org/10.21203/rs.3.rs-8619471/v1
- Primary Topic
- Emotion and Mood Recognition
- Type
- preprint