A Monocular-LiDAR SLAM Guided by Image Segmentations

Robust, accurate fusion simultaneous localisation and mapping (SLAM) remains a key research area despite recent progress. Monocular camera struggles with scale recovery, and LiDAR (Light Detecting and Ranging) faces challenges in feature extraction. This study proposes a method that fuses LiDAR and a monocular camera, guided by a deep learning network for robot platform. By segmenting images and grouping point and plane features, the method gives depth to point features and improves pose estimation and optimisation. Tested on the KITTI and HILTI datasets, it achieved an average error of 0.91% on the KITTI and successfully recovered scale for monocular SLAM in the HILTI without compromising accuracy and achieved better robustness in the HILTI dataset.