Transfer learning-based facial emotion detection for enhanced driver safety: a hybrid approach

Abstract In this article, we propose a novel hybrid approach for facial emotion recognition aimed at enhancing on-road driver safety. This hybrid method combines a pre-trained Residual Network (ResNet50) with two additional convolutional layers, enabling more refined feature extraction by integrating both deep pre-learned features and task-specific spatial patterns. This tailored architecture addresses key challenges in real-world applications by leveraging transfer learning to reduce training time while maintaining high accuracy—crucial for real-time safety systems. The proposed model is evaluated against three baseline models: a custom-designed Convolutional Neural Network, the standard ResNet50, and the pre-trained ResNet50, using the Facial Expression Recognition 2013 dataset. It achieves superior performance, with an accuracy of 99.33%, a precision of 99.40%, a recall of 99.07%, and an F1-score of 99.23%. Furthermore, testing on the Keimyung University Facial Expression of Drivers dataset, which captures driver emotions under varied lighting and occlusion conditions, demonstrates the model’s robustness and adaptability. This work contributes a high-performance, computationally efficient solution to the field of driver emotion recognition, with strong potential to improve road safety in real-time environments.