Emotion Analysis During Driving Tasks Using Facial Landmarks

Driver stress has a significant impact on cognitive function and driving performance, which increases the risk of an accident. Stress, which includes environments like congestion, monotonous conditions, and tailgating elicits distinct emotions that affect the driver’s psychological and physiological states. This study looks into the relationships between facial expression classification using the k-means, physiological indicators, and subjective evaluations under stress-related conditions. Facial features related to eye, mouth, and head movements were extracted using MediaPipe, and expressions were divided into four clusters using the elbow method. The physiological evaluation involved analyzing heart rate variability using the LF/HF ratio and nHF as indicators of sympathetic and parasympathetic nervous activity, respectively. Subjective evaluations were based on Russell’s circumplex model, which used a Visual Analog Scale (VAS) to quantitatively assess emotions along the “pleasure-unpleasure” and “arousing-sleepy” dimensions. Three participants drove in virtual environments that replicated congestion, monotony, and tailgating. The results showed that k-means classification accurately captured physiological changes, with monotonous driving exhibiting increased LF/HF and decreased nHF, which corresponded to cluster transitions over time. Furthermore, subjective evaluations revealed psychological differences between clusters that were not evident from physiological indicators alone. These distinctions, such as increased sleepiness in certain clusters, highlight the complementary roles of both methods in capturing complex emotional states. This study demonstrates the significant potential for combining k-means-based facial expression classification with physiological and subjective metrics to comprehensively assess driver emotions. By combining these approaches, this method provides useful insights for developing advanced driver assistance systems with the goal of improving emotion prediction accuracy and overall road safety.