UAV Visual Localization via Integrated Steerable Semantic Feature Learning and Density-Based Clustering

Unmanned aerial vehicles (UAVs) visual localization capabilities provide a promising solution for ensuring reliable navigation, especially in environments where Global Navigation Satellite System (GNSS) signals are obstructed or unavailable. It can be implemented without reliance on external signals or additional equipments. However, significant changes in the appearance of the ground surface, as well as variations in illumination, rotation, and viewpoint, make the matching between UAV imagery and reference satellite images challenging. In this study, a UAV visual localization method based on deep learning features with steerable semantic information and density-based clustering is proposed to enhance the robustness and accuracy of localization. The proposed lightweight semantic-aware steerable feature extraction network (SemSNet) integrates a multilevel reduced semantic segmentation block (MR-SSB) to extract local features with rotation-invariant semantic and structural information. MR-SSB learns pixel-level semantic features domain adaptively by leveraging semantic information from an off-the-shelf semantic segmentation network combined with a semantic label remapping technique. During the matching phase, a hybrid density-based clustering integrating multiple Gaussian models (MGMDBC) identifies corresponding reference regions to maximize covisibility. Experiments conducted on public and self-collected challenging UAV datasets demonstrate that our method can effectively overcome severe changes in viewpoint and ground surface. Our method achieves a minimal average localization error of under 5 m and a maximum improvement of 6.38 m in accuracy compared to previous models.