An interactive approach to extrinsically calibrate 3D LiDAR and monocular camera using open source toolchain

In many fields like robotics and autonomous driving, Light Detection And Ranging (LiDAR) and camera sensors complement each other. LiDAR and camera have their strengths and weaknesses in environmental perception. Any transformation of information from LiDAR to camera and vice versa requires accurate extrinsic calibration. This paper addresses the problem of extrinsic calibration between a 3D LiDAR and a monocular camera using a target-based method. The tool design has interactive interface that is implemented using an open source toolchain and can deal with high as well as low resolution LiDARs and cameras. The method utilizes point-to-point correspondence between camera pixels and LiDAR points from an Augmented Reality University of Cordoba (ArUco) marker target, which is in the viewpoint of both sensors. Pixel coordinates of the ArUco marker in the camera frame are detected automatically. The LiDAR points are selected by segmenting the marker surface, and then through a series of steps, the corner and center point features of the LiDAR are calculated. This procedure is repeated for different positions of the marker, then a small subset of inliers are filtered to compute rotation and translation matrix through the optimization algorithm. The tool can consistently achieve reprojection errors of less than one pixel. The code and demo dataset are released as open source for public use. Following is the link for the code https://github.com/tui-abdul/3d_lidar_camera_calib.