EDPCS: Efficient Dynamic Gesture Recognition under Point Cloud Sequences

Dynamic gesture recognition is an important part of virtual reality and human-computer interaction, while point clouds provide important clues for gesture recognition because they contain rich spatial information. However, dynamic gesture recognition under point cloud sequences is still facing challenges such as difficulty capturing spatio-temporal correlation and insufficient global feature integration capability. To address these challenges, in this paper, we describe gesture recognition as an irregular sequence recognition problem, with the goal of capturing spatio-temporal correlations between point cloud sequences and enhancing the salient features at certain locations in the sequences, and we propose a novel and effective method, EDPCS, which focuses on intra- and inter-frame feature extraction by introducing self-attention mechanism and cross-attention mechanism to enhance the model's spatio-temporal feature representation and improving the integration of global features to efficiently recognize dynamic gestures. Firstly, while maintaining the spatial structure, our method captures the global temporal dynamic features between point cloud frames through the self-attention mechanism in the intra-frame feature extraction stage; then, in the inter-frame feature extraction stage, the features of the neighbouring frames are fused into the current frame through the cross-attention mechanism, thus enhancing the feature extraction effect. This approach is not only effective in recognizing dynamic gestures but can also be integrated into many other sequence learning methods. In terms of gesture recognition, our method improves the classification accuracy from 86.1% to 88.6% on the NVGesture dataset and from 87.8% to 94.5% on the SHREC'17 Track dataset. The ablation experiments further validate the effectiveness of the self-attention mechanism and the cross-attention mechanism in our approach. Our study demonstrates that the dynamic gesture recognition method based on self-attention and cross-attention mechanisms for point clouds has important application potential and provides new ideas and methods for the field.