Self-Supervised Tight Coupling of GNSS with Neural Radiance Fields for UAV Navigation

Unmanned Aerial Vehicles (UAVs) and urban air mobility (UAM) technologies are revolutionizing transportation and service delivery in urban environments. However, these environments pose significant challenges for UAV navigation due to the limitations of single-sensor localization solutions. Traditional GNSS-based positioning is often degraded in urban canyons due to signal blockages and reflections, while current deep-learning methods require extensive labeled datasets and computational resources. To address these challenges, we introduce a novel self-supervised approach that tightly couples GNSS measurements with Neural Radiance Fields (NeRF) for improved UAV localization in complex urban settings. Our method leverages the strengths of NeRF to create detailed, flexible 3D maps of urban environments, which are then used for visual corrections to GNSS measurements. We use a self-supervised learning framework that combines semantic and structural image-matching techniques to align UAV camera images with NeRF-rendered views. This approach addresses the challenges of limited labeled data and complex error propagation across different sensor modalities. We validate our method using a real-world urban UAV dataset, demonstrating superior performance compared to traditional GNSS-only solutions and other fusion techniques, particularly in scenarios with degraded GNSS signals. Our framework offers a promising solution for enhancing UAV navigation accuracy and reliability in challenging urban environments, with numerous applications in autonomous aerial systems and urban mapping.