In-Depth Analysis of Diverse Driver Behaviors using Hybrid Multimodal Machine Learning

Analyzing diverse and intricate drivers’ behavior is a complex task when there is variation, modalities and heterogeneity in the datasets. SIMUSAFE dataset, encompassing scenario, vehicular, neurophysiological, and video analysis data, all describing the behavior of road users. This also characterizes the heterogeneity in drivers’ behavior in terms of risk and hurry, using both real-time on-track and in-simulator driving. This study focuses on a hybrid approach of multimodal machine learning (MML) to scrutinize drivers’ behavior, i.e., categorizing based on personalized criteria for conventional and negative driving. Here, the hybrid MML comprise unsupervised machine learning i.e. K-Means and Spectral clustering techniques to uncover hidden structures within the dataset, and subsequently using supervised machine learning i.e. Random Forest to enhance comprehension. Besides, an exploratory experiment is conducted on the heterogeneous data, hybrid MML helps to scrutinize drivers’ behavior as 'conventional' or 'negative'. According to the analysis, unveiled distinct structures shedding light on conventional and negative driving behaviors. Negative driving is defined by features such as violations, gaze, risk source, and distraction, while conventional driving encompasses the rest.