LIVS: a tightly coupled LiDAR, IMU, and camera-based SLAM system under scene degradation

Abstract In increasingly complex environments, single-sensor-based SLAM systems face scene degradation problems such as strong light, dynamic obstacle interference, and lack of structural features in the environment, which severely constrain the localization accuracy and mapping effectiveness of SLAM systems. To address this problem, this paper proposes a tightly coupled laser-IMU-visual SLAM system (LIVS) based on factor graph optimization. LIVS consists of two subsystems: the laser inertial system (LIS) and the visual inertial system (VIS). The pose estimation of the LIS can provide an initialization for the VIS; LiDAR suffers scene degradation in environments lacking structural features, while the VIS provides better constraints for Lidar in this environment. Firstly, a method of selecting keyframes based on ORB features and the relative motion of the mobile robot is proposed as a means of improving the positioning accuracy and tracking ability of the system in the case of strenuous motion, and a strategy of local sliding window optimization is adopted to effectively improve the real-time performance and robustness of the system. Meanwhile, in order to further improve the global map consistency and positioning accuracy, a hybrid closed-loop detection method based on laser and vision is proposed, which utilizes the geometrical features of laser and the image features of vision to accomplish the hybrid closed-loop detection. Secondly, the laser inertia factor, IMU pre-integration factor, vision inertia factor, and closed-loop factor are globally optimized in a factor map framework to achieve accurate estimation of the robot pose. Finally, this paper presents experimental evaluations on both public and homemade datasets and compares the performance of the proposed algorithm with that of other algorithms. Experimental results show that the algorithm in this paper exhibits better performance in both localization accuracy and mapping effectiveness.