Driver Activity Recognition With Vision Transformer Using Time–Frequency Representations Derived From Wrist-Worn Sensors

Driving activity recognition is crucial for improving road safety and preventing accidents, as distracted driving is a major cause of traffic incidents. Recently, driver status monitoring systems using wrist-worn sensors have been developed as small, low-cost, and user-friendly approaches to support safe driving. Although previous studies have explored driver activity recognition using wearable sensors, most have focused on analyzing time-series sensor data obtained through traditional statistical models. Subsequent studies revealed that frequency-domain analysis can provide additional informative features that enhance recognition performance. Therefore, this study investigated the extent to which recognition performance could be improved by incorporating time–frequency information and deep learning models. Time–frequency features were derived using a short-time Fourier transform (STFT) or wavelet transform, and a vision transformer (ViT) was employed as a deep learning architecture. Consequently, our approach achieved superior performance compared with previous studies. Furthermore, to address the challenge of limited data availability in real-world driving scenarios, we evaluated the model under small-sample conditions and demonstrated its effectiveness using pretrained models. Experiments using a subset of sensor channels identified a configuration that resulted in minimal performance degradation, even with a limited number of channels.