Hand-object interaction reconstruction method based on diffusion model and SDFs

In this paper, we propose a novel hand-object interaction reconstruction method based on diffusion model and signed distance fields (SDFs). This method uses a multi-scale feature extractor, an attention bridge, a segmentation branch module based on vector merging, and reconstructs the 3D structure of hand and object from monocular color RGB images. In this study, a multi-scale feature extractor is used to extract image features, and the feature representation ability is enhanced by the attention mechanism. In addition, a diffusion model is introduced into the hand and object shape estimation in HOI to optimize the feature extraction, and the Marching Cubes algorithm is used to extract zero isosurfaces from the signed distance field and convert them into grid form to obtain the reconstruction result of the three dimensional surface. Through ObMan data set, we verified the HOI method, and the reconstruction results showed obvious improvement in detail and quality.