Research on a Safe Driving Monitoring System Based on Multimodal Data

Abstract To address the safety issues in construction machinery caused by abnormal driving behaviors, this paper proposes a Multimodal Safe Driving Monitoring System (MSDMS). The system consists of two main components. The first is the Multimodal Sentiment Analysis model based on Hybrid Attention (MSA-HA), designed for emotional driving detection. By integrating hybrid attention mechanisms with a Tensor Fusion Network, this model enhances multimodal feature interaction and improves the accuracy of emotion detection. The second component employs the YOLOv8n-DPCLN model for the detection of fatigue and distracted driving. In this model, a Deepwise Partial Convolution Layer Network (DPCLN) replaces the bottleneck within the C2f module to enhance overall performance. Experimental results demonstrate that the MSA-HA achieves binary and ternary classification accuracies of 83.5% and 74.76%, respectively, on the SIMSv2 dataset, outperforming other mainstream models. Compared to YOLOv8n, the YOLOv8n-DPCLN improves [email protected] and [email protected]–0.95 by 1.9% and 3.7%, respectively, while reducing the parameter count by 0.4 M and computational cost by 1.1 GFLOPs.Finally, MSDMS achieves recognition rates of 78.7%, 93%, and 86% for emotion, fatigue, and distracted driving, respectively, thereby satisfying the requirements for safe driving monitoring.

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-17
DOI
https://doi.org/10.21203/rs.3.rs-8619471/v1
Primary Topic
Emotion and Mood Recognition
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Research on a Safe Driving Monitoring System Based on Multimodal Data

Lingling Tian, Zhang Dengyin, hao Cui, Zepeng Wei et al.
5 links
Research Square
Emotion and Mood Recognition
preprint

Research on a Safe Driving Monitoring System Based on Multimodal Data

Lingling Tian, Zhang Dengyin, hao Cui, Zepeng Wei, Jie Cai
preprint en

Abstract

Abstract To address the safety issues in construction machinery caused by abnormal driving behaviors, this paper proposes a Multimodal Safe Driving Monitoring System (MSDMS). The system consists of two main components. The first is the Multimodal Sentiment Analysis model based on Hybrid Attention (MSA-HA), designed for emotional driving detection. By integrating hybrid attention mechanisms with a Tensor Fusion Network, this model enhances multimodal feature interaction and improves the accuracy of emotion detection. The second component employs the YOLOv8n-DPCLN model for the detection of fatigue and distracted driving. In this model, a Deepwise Partial Convolution Layer Network (DPCLN) replaces the bottleneck within the C2f module to enhance overall performance. Experimental results demonstrate that the MSA-HA achieves binary and ternary classification accuracies of 83.5% and 74.76%, respectively, on the SIMSv2 dataset, outperforming other mainstream models. Compared to YOLOv8n, the YOLOv8n-DPCLN improves [email protected] and [email protected]–0.95 by 1.9% and 3.7%, respectively, while reducing the parameter count by 0.4 M and computational cost by 1.1 GFLOPs.Finally, MSDMS achieves recognition rates of 78.7%, 93%, and 86% for emotion, fatigue, and distracted driving, respectively, thereby satisfying the requirements for safe driving monitoring.

Research Square
Nanjing University of Posts and Telecommunications (CN)
Sustainable cities and communities, Industry, innovation and infrastructure
Emotion and Mood Recognition
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon