Fusing thermal and visual sensory data for enhanced VIO localization in low-light conditions

Abstract Visual inertial odometry (VIO) is a crucial technique for autonomous driving, but it encounters challenges in dark or shadowy environments, where the accuracy of the positioning system may be degraded or lost. Moreover, different kinds of sensor noise can affect the positioning accuracy. In this paper, we present a novel VIO localization algorithm that fuses data from a thermal camera, a visual camera, and an inertial measurement unit (IMU) in a tightly coupled manner. To handle non-Gaussian noise in dark environments and improve positioning accuracy and stability, we employ an iterative extended Kalman filter (IEKF) with the maximum correntropy criterion and adaptive noise, enhancing the robustness. We first combine the thermal camera with the conventional VIO method to form a thermal VIO framework. Then, an adaptive maximum correntropy criterion IEKF (AMCCIEKF) algorithm is proposed to refine the accuracy and stability. We implement and experimentally evaluate our proposed localization method using an open dataset. In a dark and shaking environment, our method reduces errors by approximately 56% compared to IEKF. Moreover, our method achieves a 41% reduction in median and 65% in standard deviation (S.D.).