Detecting Traffic Police Hand Gestures With Movenet Thunder for Self-Driving Cars

This study introduces a novel Traffic Police Hand Gesture Recognition System specifically designed for autonomous vehicles in India, utilizing TensorFlow's MoveNet Thunder model. In Indian urban environments, where traffic police frequently use hand gestures to manage traffic, autonomous vehicles face significant challenges in accurately interpreting these signals. This paper presents the development of a deep learning-based system that recognizes and interprets traffic police gestures such as 'Stop cars from the left and right', 'stop cars from the front', 'stop cars from the back', 'start cars from the left', 'start cars from the right', 'Vehicles should be started at <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$T$</tex>-Point', 'Stop cars from the front and rear', 'Unidentified Position'. The methodology involves creating a custom dataset of approximately 8,000 labeled images of traffic police performing gestures under varied environmental conditions, which was used to train the MoveNet Thunder model. The system relies on camera-based inputs for realtime gesture detection and processing. Extensive testing in the Carla simulator demonstrated the system's effectiveness, achieving an accuracy of 89% in gesture recognition. The results indicate significant improvements in the interpretability and reliability of autonomous vehicle responses to manual traffic signals, contributing to safer and more efficient vehicle navigation in environments with human-directed traffic control. This research lays the groundwork for future applications of gesture recognition in autonomous vehicles, particularly in complex urban traffic settings.