Hand Geometry Reconstruction and Rendering Using 3D Gaussian Splatting Based on KL Divergence Optimization

This paper presents a novel hand reconstruction framework—3D Gaussian Splatting Hand Reconstruction (3D-GSHR), which integrates 3D Gaussian splatting with MANO model to achieve real-time, photorealistic 3D hand reconstruction. By incorporating the existing MANO model and adding geometric priors, we not only enhance the resolution representation of hand geometry but also achieve efficient deformation from the canonical space to the target pose, overcoming the limitations of traditional MANO model subdivision. The challenge of achieving realistic rendering of gesture actions from arbitrary viewpoints using monocular hand videos is complex. To address this, our framework employs 3D Gaussian Splatting (3DGS) to reconstruct both the visual appearance and geometric structure of hand movements, enabling realistic rendering of gesture animations from multiple perspectives with rapid training and rendering processes. This method contrasts with existing NeRF-based models, which require extensive training and long rendering times per frame. To further improve texture modeling, we use KL divergence to refine Gaussian density control, accelerating network training. Additionally, our approach simultaneously optimizes geometry and appearance models in an end-to-end training process. Experiments on the InterHand2.6M dataset show that our method achieves commendable performance both quantitatively and qualitatively, with swift training and rendering speeds.