SPTL-LCC: Single-Shot, Pixel-Level, Target-Free, and LiDAR-Type Agnostic LiDAR-Camera Extrinsic Self-Calibration

The fusion of LiDAR and camera data holds vast application prospects, with calibration as a crucial prerequisite. In this paper, we propose SPTL-LCC, a LiDAR -type agnostic and target-free LiDAR-camera extrinsic calibration framework. The core of SPTL-LCC lies in extracting accurate 3D-2D correspondences in 2D image space using a densified point cloud and an image. To ensure the acquisition of a densified point cloud, a point cloud accumulation evaluation metric is proposed. Building upon this, a multi-stage LiDAR image processing and optimization method based on a virtual camera is proposed. Then, multiple image matching networks are employed to estimate 2D-2D correspondences between the LiDAR images and camera images, resulting in 3D-2D correspondences. A progressive reprojection error minimization algorithm is subsequently proposed to optimize the extrinsic parameters based on these correspondences. SPTL-LCC improves LiDAR image quality in a few iterations, enhancing correspondence estimation accuracy, thus achieving coarse-to-fine calibration. Extensive experiments on four self-recorded datasets show SPTL-LCC outperforms state-of-the-art methods in terms of robustness, accuracy, and practicality, with an average error of less than 0.03 m in translation and 0.4<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{\circ }$</tex-math></inline-formula> in rotation. Moreover, experiments on the KITTI dataset show SPTL-LCC even achieves comparable performance to end-to-end methods trained on this dataset, which exhibit poor generalization. Our code and datasets are open-source: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/NKU-MobFly-Robotics/SPTG-LCC</uri>.