Human novel pose generation with pose optimization and importance sampling based on NeRF

The Neural Radiance Field (NeRF) method has emerged as a groundbreaking technique for human reconstruction, enabling the generation of high-quality, photorealistic rendering of reconstructed objects. Despite its promising results, several challenges persist, particularly when it comes to generalizing to unknown poses during the generation of virtual human animations. This paper proposes a method for generating and optimizing novel human poses based on NeRF. Given that existing pose driven NeRF methods heavily rely on the accuracy of SMPL body parameters, we implemente human pose optimization based on motion capture data as prior knowledge, resulting in more accurate SMPL parameters, which contributes to a more precise human canonical model. Additionally, we propose a pose-driven deformation field based on linear blend skinning, combining blend weight fields with the 3D human skeleton to achieve precise mapping from observed poses to the canonical pose. By estimating the camera ray transmittance distribution during volume rendering, we propose importance sampling method avoiding sampling points in free space, which further improves rendering quality. Extensive experiments demonstrate that our method yields significant improvements in rendering quality for both training poses and novel poses, as evidenced by results on the ZJU-MoCap dataset and our custom dataset. This work not only advances the capabilities of NeRF in human reconstruction but also opens new avenues for realistic virtual human animation.