Depth Completion Algorithm Based on Panoptic Segmentation Assistance

The depth completion task aims to solve the problem of sparse or missing depth values caused by the sensor itself or the external environment. This paper proposes an RGB camera/LiDAR fusion algorithm based on image-guided depth information completion. Initially, sparse radar point clouds are converted into sparse depth maps, followed by the application of an image segmentation algorithm to the RGB images to generate Boolean masks. Guided by these masks, the study employs a depth completion algorithm based on morphological filtering to densify the sparse depth maps into dense depth maps, thus creating fusion information. The advanced nature of the proposed image-guided depth inflation algorithm in maintaining edge depth discontinuities was validated through testing on the KITTI depth completion dataset. This algorithm effectively preserves depth information at image edges, reducing the root mean square error and mean absolute error by 2.3% and 1.2%, respectively, compared to non-image-guided methods. Back-projection experiments shows that this method can enhance the quality of 3D reconstruction.