3D Pose Estimation Using Virtual Projection Based on 3D Reconstructed Model

In this paper, we estimate and refine 3D human pose using the 3D point cloud or mesh model reconstructed from RGB-D cameras or volumetric capture systems. We first reconstruct the 3D model using the multi-view cameras to estimate a highly accurate skeleton. To obtain a 2D skeleton with low error, the reconstructed 3D model is projected to four virtual planes after decidi ng the direction of the 3D model. Four 2D skeletons are estimated from four images projected in the virtual plane. Afterward, the refinement process selects candidate joints based on the distribution of local vertices and the DBSCAN algorithm. It applies a sphere fitting to ensure that the final joints are located within the body volume. The joints are combined at the intersection through the back-projection of the joints, including those in the 2D skeleton on the virtual plane. The joints in the intersection are refined using the spatial distribution of the 3D information. Through the proposed method, we estimated a stable and geometrically consistent 3D human pose from reconstructed volumetric data. Using models with ground truth, we calculated the MPJPE between the skeletons of the proposed and the ground truth. The 3D pose estimation was evaluated through a visual assessment of the captured image, and the results were quantitatively compared with the 3D joint positions acquired by the motion capture device.