Long-Duration UAV Localization Across Day and Night by Fusing Dual-Vision Geo-Registration with Inertial Measurements

Remote sensing visual-light spectral (VIS) maps provide stable and rich features for geo-localization. However, it still remains a challenge to make use of VIS map features as localization references at night. To construct a cross-day-and-night localization system for long-duration UAVs, this study proposes a visual–inertial integrated localization system, where the visual component can register both RGB and infrared camera images in one unified VIS map. To deal with the large differences between visible and thermal images, we inspected various visual features and utilized a pre-trained network for cross-domain feature extraction and matching. To obtain an accurate position from visual geo-localization, we demonstrate a localization error compensation algorithm with considerations about the camera attitude, flight height, and terrain height. Finally, the inertial and dual-vision information is fused with a State Transformation Extended Kalman Filter (ST-EKF) to generate long-term, drift-free localization performance. Finally, we conducted actual long-duration flight experiments with altitudes ranging from 700 to 2400 m and flight distances longer than 344.6 km. The experimental results demonstrate that the proposed method’s localization error is less than 50 m in its RMSE.