Calibration between a panoramic LiDAR and a limited field-of-view depth camera

Depth cameras and LiDARs are commonly used sensing devices widely applied in fields such as autonomous driving, navigation, and robotics. Precise calibration between the two is crucial for accurate environmental perception and localization. Methods that utilize the point cloud features of both sensors to estimate extrinsic parameters can also be extended to calibrate limited Field-of-View (FOV) LiDARs and panoramic LiDARs, which holds significant research value. However, calibrating the point clouds from two sensors with different fields of view and densities presents challenges. This paper proposes methods for automatic calibration of the two sensors by extracting and registering features in three scenarios: environments with one plane, two planes, and three planes. For the one-plane and two-plane scenarios, we propose constructing feature histogram descriptors based on plane constraints for the remaining points, in addition to planar features, for registration. Experimental results on simulation and real-world data demonstrate that the proposed methods in all three scenarios achieve precise calibration, maintaining average rotation and translation calibration errors within 2 degrees and 0.05 meters respectively for a $$360^{\circ }$$ linear LiDAR and a depth camera with a field of view of $$100^{\circ }$$ vertically and $$70^{\circ }$$ degrees horizontally.