3D reconstruction of hand object based on a single RGB image: integrating anomaly inference and optimizing attention mechanisms

Estimating the pose of hand-object interactions from a single RGB image is a complex task because self-obscuration of the hand and occlusion between hand-objects can seriously affect the accuracy of pose estimation. To cope with the above problems, we construct a new network framework to extract hand-object features using a single backbone network with an added anomaly inference module, which solves the problem of unclear hand-object features during feature extraction and enhances the usability of the features. Furthermore, we enhance the attention mechanism within the Transformer model, which combines the accuracy of self-attention with the computational efficiency of linear attention. Results show that our proposed network framework attains significant substantial improvements on both HO3D and Dex-Ycb datasets.