Assessment of Driver Stress using Multimodal wereable Signals and Self-Attention Networks

Abstract Assessment of driver stress, crucial for road safety, can greatly benefit from the analysis of multimodal physiological signals. However, fusing such heterogeneous data poses significant challenges, particularly in intermediate fusion where noise can also be fused. In this study, we address this challenge by exploring a 1D convolutional neural network (CNN) with self-attention mechanisms on multimodal data. Electrocardiogram (ECG) signals (256 Hz) and respiration (RESP) signals (128 Hz) were obtained from ten subjects using textile electrodes while driving in different scenarios, namely normal driving and phone usage (calling). The obtained multimodal data is preprocessed and then applied to a self-attention mechanism (SAM) CNN (SAMcNN) to identify driver stress. Experiments are validated using Leave-one-outsubject cross validation. The proposed approach is capable of classifying driver stress. It is observed that shorter segments yield an accuracy of 64.16% compared to longer segment lengths. Thus, exploring self-attention mechanisms for multimodal signals using wearable shirts facilitates non-intrusive monitoring in real-world driving scenarios.