IVAH: Invisible-Region Augmented Human Rendering From Monocular Images

Most neural radiance field(NeRF)-based human rendering methods rely on per-subject optimization or multi-view data input to reconstruct the 3D radiance field of human subjects. However, optimization-based approaches lack the flexibility to render multiple subjects with a single model, while acquiring multi-view data necessitates a calibrated multi-view setup. To overcome these limitations, monocular image-based methods have been introduced to enable the rendering of novel views and poses of human subjects in a feed-forward manner from a single image. Nonetheless, monocular images inherently provide limited information, presenting challenges when rendering occluded or invisible regions of the subject. To address this limitation, we propose InVisible-region Augmented Human rendering (IVAH), a method that generates and leverages additional appearance data to represent the unseen regions of the input observation image. Specifically, IVAH synthesizes a back image and a UV map of the human subject, extracting features that facilitate the rendering of the occluded areas. Experimental results demonstrate that the inclusion of generated appearance data significantly enhances the rendering of invisible regions. Furthermore, our findings indicate that IVAH surpasses current monocular image-based techniques in rendering performance.