HandNet: A Robust Ensemble Framework for Hand Keypoint Detection in Diverse Scenarios

Hand landmarks detection and pose estimation are crucial in various applications such as rehabilitation, assistive technology, and human-computer interaction. In this work, we propose a novel lightweight ensemble-based hand landmarks detection model called HandNet, which is designed for real-time applications. Our model is trained and evaluated on the FreiHand dataset, achieving a Mean Per Joint Position Error (MPJPE) of 7.80 mm and a Percentage of Correct Keypoints ([email protected]) of 99.5%, surpassing existing state-of-the-art methods. Our proposed model employs an ensemble of four convolutional neural networks (EfficientNet-B0, ResNet-50, MobileNet-V3, and DenseNet-121) as backbones for efficient landmark detection with a computationally efficient pipeline. We conduct extensive quantitative evaluations comparing our model with existing SOTA models (LightWeightNet, LGCANet-w18, and MobileNet-v2), demonstrating superior accuracy. Our proposed approach is particularly beneficial for real-time applications where highly accurate hand landmark tracking is essential.