A lightweight and efficient gesture recognizer for traffic police commands using spatiotemporal feature fusion

In response to the demand for efficient and accurate recognition of traffic police gestures by driverless vehicles, this paper introduces a novel traffic police gesture recognition framework (Novel Traffic Police Gesture Recognizer, NTPGR). Initially, keypoints related to traffic police gestures are extracted using the Efficient Progressive Feature Fusion Network (EPFFNet), followed by feature modeling and fusion to enable the recognition network to better learn the temporal characteristics of gestures. Additionally, a convolution network branch and a hybrid attention branch are incorporated to further extract skeleton information from the traffic police gesture data, assign different temporal weights to key frames, and enhance the focus on important channels. Finally, in conjunction with Long Short Term Memory (LSTM), a multi-branch gesture recognition network, termed the Multi-Sequence Gesture Recognition Network (MSNet), is proposed to facilitate the integration of three branches of gesture features, thereby enhancing the targeted extraction of temporal characteristics in traffic police gestures. Experimental results indicate that NTPGR achieves 97.56% and 96.76% accuracy on the Police Gesture Dataset and UTD-MHAD Dataset, respectively, as well as average response times of 0.76s and 0.74s. It not only recognizes traffic police gestures in real-time with high efficiency but also demonstrates strong robustness and Credibility in recognizing gestures in complex environments and dynamic scenarios.