YOLOv5 and LSTM-Based Driver Drowsiness Detection with Emergency Protocols for Smart Vehicles

An intelligent driver monitoring and hazard detection system that utilizes YOLOv5, a deep learning object detection model, and Long Short-Term Memory (LSTM) networks has been developed to escalate the safety of vehicles. The system uses YOLOv5 for facial recognition and monitoring of the environment around the vehicle, which includes detection of potholes, pedestrians, and erratic motor vehicle movements. Moreover, LSTM analyses the movement patterns of the driver's mouth and eyes to track possible signs of drowsiness or other medical conditions. Once a danger is detected, either from the environment or posed physiologically, the system puts in place measures that include alerting other vehicles and notifying emergency personnel through IoT channels. The system utilizes a blend of standard and customized datasets for training, taking into consideration different weather conditions to improve robustness. Evaluation tests of the model indicate it has achieved a validation accuracy of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{9 2 \%}$</tex> in estimating with <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\mathbf{9 4 \%}$</tex> accuracy during daytime hours, while over 85% of the time in foggy or rainy conditions. The system also reduces the false positive rate to 6.4% while improving emergency response accuracy to 96% which outperforms predecessor systems in both reliability and responsiveness. This two-fold approach system provides an enhanced perspective toward safety in smart vehicles. Expanding this work in the future may include the integration of biometrics devices, predictive analytics, and application into more complex autonomous vehicular systems which may be useful for thorough testing and evaluation.