Perspective‐Aware Fusion of Incomplete Depth Maps and Surface Normals for Accurate 3D Reconstruction

ABSTRACT We address the problem of reconstructing 3D surfaces from depth and surface normal maps acquired by a sensor system based on a single perspective camera. Depth and normal maps can be obtained through techniques such as structured‐light scanning and photometric stereo, respectively. We propose a perspective‐aware log‐depth fusion approach that extends existing orthographic gradient‐based depth‐normals fusion methods by explicitly accounting for perspective projection, leading to metrically accurate 3D reconstructions. Additionally, the method handles missing depth measurements by leveraging available surface normal information to inpaint gaps. Experiments on the DiLiGenT‐MV data set demonstrate the effectiveness of our approach and highlight the importance of perspective‐aware depth‐normals fusion.