Driver Drowsiness Detection Using Multi-Frames Approach with Vision Transformer Models

Driver drowsiness is a major contributor to road accidents, necessitating the development of effective real-time monitoring systems to enhance road safety. This study presents a multi-frame approach for driver drowsiness detection, improving upon conventional single-frame detection by capturing temporal dependencies in risky driving behaviors such as drowsiness, nodding, and yawning. The proposed Vision Transformer models proposed in this work are ViT B16 and Swin V2S, where the multiframe approaches on these models are implemented. A dataset collected from 44 individuals expressing normal, yawning and nodding behaviors is utilized, with augmentation techniques applied to enhance class balance during training. Experimental results demonstrate that a 5 -frame approach with the Swin V2S model achieves the best detection performance, with a training accuracy of 97.05 % and a test accuracy of 97.07 %. The optimized model is then deployed on both PC and Raspberry Pi 5, the latter being a cost-effective and portable solution with inherent computational constraints. Benchmarking results indicate that Raspberry Pi 5 achieves an inference time of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$0.6075 \mathbf{sec} /$</tex> frame for single-frame processing and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$0.5734 \mathbf{sec} /$</tex> frame for 5-frame processing using Swin V2S model, significantly slower than a PCbased implementation which achieves <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$0.0059 \mathbf{sec} /$</tex> frame inference for all tested processing methods using ViT B16. These findings highlight the feasibility of deploying ViT-based drowsiness detection on low-power edge devices, while emphasizing the tradeoffs between portability and inference speed.