An Efficient and Lightweight Measurement Strategy for Dynamic Gesture Recognition via Frequency-Time Fusion

Abstract Dynamic gesture recognition technology for wearable devices is a key enabler of efficient human-machine interaction within the embedded devices. However, existing methods still face challenges such as low recognition accuracy in small sample conditions, high computational overhead, limited robustness, and inadequate real-time performance on embedded devices, particularly in the context of wearable devices where real-time processing and low-power consumption are crucial. This paper proposes a lightweight and adaptive dynamic gesture recognition method based on data glove input that fuses time-frequency domain features to overcome these challenges. The proposed approach integrates dynamic-weight convolution with a Gaussian kernel-optimized shared cross-attention mechanism, effectively combining both time-domain and frequency-domain features while capturing global and local information from gesture signals acquired by a custom data glove equipped with bending and attitude sensors. Additionally, parameter sharing and a lightweight parameter design are employed to reduce computational complexity. Experimental results on the "National Standard Sign Language Dictionary" dataset reveal a test-set recognition accuracy of 98.44%, a gesture recognition time of 2.5 ms per gesture, and a model size of 29.75k parameters, occupying only 0.12 MB of storage. Tests on the Jetson Nano embedded device show that the proposed method maintains a high recognition accuracy of 96.89%, with a recognition time of 5.7 ms per gesture and an average power consumption of 2.58W in low-power mode, confirming its efficiency, real-time capability, and feasibility for embedded devices, especially in resource-constrained environments.