Multi-feature fusion network: Hand pose estimation for hand-object interaction

Hand-object interaction often involves significant occlusion of the hand, making this task particularly challenging. Recently, many studies have made substantial progress in predicting hand pose despite severe occlusions, either by directly predicting hand mesh vertices or by utilizing a priori hand models. However, most of these studies overlook the implicit hand features in the occluded regions, which affects the accuracy of the results. To address this issue, we propose a 3D hand pose estimation method called the Multi-Feature Fusion Network. This network learns the correlation between hand features and object occlusion features, utilizing the occluded region information to refine the hand global features, and extracts the hand local features at the same time. Through the joint guidance of multiple features, an accurate hand pose is reconstructed. Extensive experiments on hand-object interaction datasets with severe occlusion problems (HO3D, DEXYCB, and FreiHAND) demonstrate that our approach is highly competitive.