Dynamic Hand Gesture Recognition by Hand Landmark Classification using Long Short-Term Memory

Hand gestures are a valuable modality for human-computer interaction, conveying information that can be used as input. Dynamic hand gestures, prevalent in real-world scenarios, necessitate considering temporal factors such as gesture initiation, termination, and frame sequence. A Long Short-Term Memory (LSTM) based recognition model was proposed to address this challenge. Data availability for dynamic hand gesture research is a significant hurdle. The dataset introduced by Fronteddu et al. provides 27 classes of dynamic hand gestures, serving as a suitable training resource. MediaPipe Hands, a computer vision framework, was leveraged to extract keypoints from each frame, capturing spatial features fed into the LSTM model. Experiments were conducted to determine the optimal dropout rate for the LSTM model. Results indicated that a dropout rate of 70% yielded the highest accuracy, achieving up to 98.53% validation accuracy and 99.71% test accuracy. These findings demonstrate the effectiveness of the proposed LSTM-based recognition model for dynamic hand gestures. Future research could explore integrating other deep learning techniques, such as attention mechanisms, to enhance the accuracy and robustness of dynamic hand gesture recognition systems. Additionally, investigating the application of the proposed model in real-world scenarios, such as virtual and augmented reality, would be valuable in assessing its practical utility.