Hand–Object Pose Estimation Based on Anchor Regression from a Single Egocentric Depth Image

To precisely understand the interaction behaviors of humans, a computer vision system needs to accurately acquire the poses of the hand and its manipulated object. Vision-based hand-object pose estimation has become an important research topic. However, it is still a challenging task due to severe occlusion. In this study, a hand-object pose estimation method based on anchor regression is proposed to address this problem. First, a hand-object 3D center detection method was established to extract hand-object foreground images from the original depth images. Second, a method based on anchor regression is proposed to simultaneously estimate the poses of the hand and object in a single framework. A convolutional neural network with ResNet-50 as the backbone was built to predict the position deviations and weights of the uniformly distributed anchor points in the image to the keypoints of the hand and the manipulated object. According to the experimental results on the FPHA-HO dataset, the mean keypoint errors of the hand and object of the proposed method were 11.85 mm and 18.97 mm, respectively. The proposed hand-object pose estimation method can accurately estimate the poses of the hand and the manipulated object based on a single egocentric depth image.