Fusion of near-infrared single-photon LiDAR with visible light camera

Abstract The fusion of light detection and ranging (LiDAR) and camera provides comprehensive information about the depth and color texture of objects, finding wide applications in autonomous driving, three-dimensional (3D) scene reconstruction, and other fields. However, point clouds acquired via LiDAR are susceptible to noise, impacting downstream tasks like surface reconstruction and object recognition demanding high-quality point cloud outputs from 3D imaging systems. This paper proposes a novel 3D fusion imaging technique that enhances the quality of the fused point clouds from three distinct aspects, improving the accuracy and precision of point clouds by employing matched filter method, designing a preprocessing workflow for point clouds and images to enable their effective fusion, and smoothing noise and accentuating texture features using a filter algorithm on fused information. The method of matched filtering has been employed to reconstruct object distance information from photon count-time histograms, resulting in an improvement of approximately 120% compared to the method of directly calculating the average photon flight time. The point cloud filter proposed utilizing fused information demonstrates superior denoising capabilities compared to the state-of-the-art and has the capability to cluster points with similar features to highlight the texture characteristics of the point cloud.