Self-Supervised Depth Completion With Calibration Enhancement and Normal Guidance

Depth completion aims to recover pixel-wise depth information from incomplete measurements. Due to the difficulty in obtaining ground-truth dense depths for supervision, many self-supervised depth completion methods are proposed based on photometric consistency between adjacent views. A novel self-supervised method, KBNet (Calibrated Backprojection Network), is recently proposed leveraging the calibration matrix to upgrade the 2.5D depth to 3D spatial information for more effective feature fusion. By observing that surface normal is highly related to depth recovery task and is sensitive to object boundary and depth variation, surface normal guidance is introduced into KBNet for enhancement in this paper. Specifically, a normal-specific sparse-to-dense module is designed to coarsely densify surface normal while preserving both ground and facade geometric information, and a normal-involved multi-modality fusion module is developed to fuse the surface normal information for serving as reliable depth recovery guidance. Experiments on both the KITTI (Karlsruhe Institute of Technology and Toyota Technological Institute at Chicago) and the NYUv2 (New York University) depth completion datasets demonstrate the advantages of our proposed method over KBNet and some other state-of-the-art self-supervised depth completion methods.