LiDAR-Visual-Inertial Tightly-coupled Odometry with Adaptive Learnable Fusion Weights

In this paper, we address the sensitivity of the 3D LiDAR-based localization to environmental structural ambiguity. Although existing approaches employ additional sensors, such as cameras and inertial measurement units, to account for such ambiguities, multi-sensor localization is still an open problem. Limitations are from the need to tune fusion parameters to compensate for limited ambiguity detection manually. Therefore, we propose a feature-based localization method that learns the fusion parameters using ground truth and thus supports autonomous mobile robotic systems in new locations. The method combines planar surface LiDAR features with close and far camera features, and its further advantage is an online adjustment of the feature weights based on the measured environment ambiguity. The evaluation has been performed on the existing M2DGR dataset and custom dataset with geometrical ambiguities. The proposed method is competitive to or outperforms the existing LiDAR-based methods F-LOAM and LIO-SAM and the Visual-Inertial localization method VINS-Mono. Based on the reported results, the proposed method is a vital combination of LiDAR-based and visual features.