Monocular 3D hand pose estimation based on high-resolution network

Abstract Hand pose estimation from monocular images has been widely applied in many fields. However, the depth ambiguity of the hand and occlusion or lighting change caused by hand-object interaction make the accurate pose estimation challenge. Currently, many methods increase the depth of models to improve the accuracy of hand pose estimation networks, significantly increasing the number of model parameters and computational complexity. To solve this issue, we propose an improved High-Resolution Network (HRDNet) for hand pose estimation. Our method mainly consists of four stages: image feature extraction, 2D information prediction, 3D joint prediction, and hand mesh reconstruction. For image feature extraction, to enhance the network’s multiscale representation capabilities, we propose a hand feature perception module and a lightweight basic module. For the 2D information prediction module, a hand pose estimation network is proposed to fully utilize the 2D information of the hand while predicting 2D joint heat maps. Additionally, we also propose a hand joint pose encoder to improve the prediction accuracy of key points in heat maps. Then, in the process of 3D joint prediction, a two-level cascaded pose estimation network is proposed to predict 3D joint coordinates. Finally, we take the Inverse Kinematics Network (IKNet) to regress pose and shape parameters for generating reconstructed hand mesh in the hand Model with Articulated and Nonrigid defOrmations (MANO) model. Our extensive experiments on two publicly available datasets have shown that our method achieves competitive results compared with state-of-the-art hand pose estimation methods.