Personalized Hand Gesture Recognition Using Few-Shot Learning

Current hand gesture recognition (HGR) systems require a substantial amount of labeled gesture data for training, making the data collection process costly and time-consuming. Although recent studies on few-shot gesture recognition attempt to address this challenge, they do not leverage recent advancements in few-shot learning and often employ models that are not optimized for hand gesture recognition. Furthermore, previous studies lack a fair comparison with standard supervised learning. In this paper, we propose a personalized few-shot hand gesture recognition (pfHGR) model that utilizes a real-time HGR architecture. Specifically, the pfHGR model employs a DD-Net backbone with a prototypical cosine classifier, trained using the P>M>F few-shot learning pipeline. Furthermore, we apply user-specific fine-tuning in order to evaluate the feasibility of deploying such models in real-world settings. We evaluated the per-user classification accuracy of the pfHGR model on the DHG dataset and compared its performance against a baseline HGR model trained using standard supervised learning and the same user-specific fine-tuning. Experimental results show that the pfHGR model outperforms the baseline model in 5-way 1-shot to 3-shot settings, with performance becoming comparable at the 4-shot setting. Specifically for 1-shot tasks, the pfHGR model achieves a user accuracy of 85.01±0.90%, which is 8.17 percentage points higher the baseline. As the number of support samples increases, performance improves steadily, reaching 94.53±0.49% in 4-shot tasks, which is 0.18 percentage points below the baseline, although the difference is not statistically significant. The pfHGR model is particularly effective in extreme low-data scenarios (1-3 shots), where standard HGR models are less effective.