Temporal and modal contributions to smartphone-based multimodal driving behavior classification: a comparative study of classical, deep learning, and patch-based time series transformer models

Understanding and classifying driving behavior is a critical component of modern intelligent transportation systems, with implications for traffic safety, fuel efficiency, and personalized driver support. As sensor-equipped mobile devices become increasingly pervasive, new opportunities have emerged for implementing data-driven behavior recognition systems in a cost-effective and accessible manner. This study presents a comprehensive and low-cost mobile framework for classifying driving behaviors using data collected entirely via a smartphone application. Unlike prior approaches that rely on embedded hardware, the proposed system performs all data acquisition and recording through a standard smartphone paired with a bluetooth-based on-board diagnostics II (OBD-II) adapter. The framework integrates multimodal sensor sources, including engine control unit (ECU) data, inertial motion sensors, enriched road metadata via the Overpass application programming interface (API), and environmental audio signals. Rather than isolating a single data domain, the system unifies mechanical, contextual, and behavioral dimensions to enable robust driving style analysis. Driving behavior was categorized into three classes (calm, normal, aggressive) using sliding time windows of 3, 5, 7, and 9 s. The effects of both window duration and data source composition on model performance were thoroughly evaluated. Classical machine learning models (artificial neural network (ANN), support vector machine (SVM), logistic regression (LR), Naive Bayes (NB)) based on engineered features were compared against deep learning architectures (convolutional neural network (CNN), long short-term memory (LSTM), gated recurrent unit (GRU), recurrent neural network (RNN)) trained on raw multivariate sequences. Results showed that multimodal integration substantially improved classification accuracy, with CNN achieving the highest performance. Additionally, the study incorporated patch-based time series transformer (PatchTST), a modern transformer-based architecture designed for time-series classification, across 128 experimental configurations. While CNN remained the top performer in overall accuracy, PatchTST yielded consistently stable and competitive results, particularly in long-window and feature-rich settings. Importantly, statistical analyses confirmed the significance of differences across feature sets, time windows, and model types. This included architectural parameters such as model depth and latent dimensionality, evaluated through analysis of variance (ANOVA) and post-hoc Tukey’s honestly significant difference (HSD) tests. By enabling high-accuracy driving behavior classification using only smartphone-based sensing, this study contributes a practical and scalable solution. The inclusion of attention-based PatchTST modeling further extends the methodological breadth, highlighting the role of transformer architectures in multivariate time-series analysis. Collectively, these contributions underscore the feasibility of deploying robust and intelligent driver monitoring systems in real-world environments.