Vision‐Based Runway Detection and Localization for Aircraft Landing in Global Navigation Satellite System‐Denied Environments

ABSTRACT To improve auto‐landing safety for fixed‐wing aircraft, precise positioning and navigation are crucial for accurately determining the aircraft's orientation and distance from the runway. Unmanned aerial vehicles (UAVs) rely heavily on the global navigation satellite system (GNSS) as a primary sensor to provide navigation information. The GNSS signal can be weak and unavailable in certain environments, making it susceptible to jamming and spoofing. This can be a problem, especially during the landing phase when the GNSS signal may not be available. With advancements in deep learning technology, vision‐based methods have become popular as alternative navigation systems for UAVs. This study focuses on developing an alternative navigation method that integrates deep learning and vision‐based techniques to provide orientation and position information for the landing of fixed‐wing aircraft. The proposed method includes two parts: runway detection and aircraft localization. The learning‐based methods, including the Mask region‐based convolutional neural network (Mask R‐CNN) and You Only Look Once (YOLO), are adopted to detect the runway and provide instance segmentation for localization. The Plücker method and convolutional neural network (CNN) regression are used to estimate the reliable position of the aircraft relative to the target runway. Since the estimated displacements from the Plücker method are highly sensitive to the slope angles of the runway edge lines, especially for the longitudinal displacement, a CNN regression is utilized to perform smoothing of the raw data. Both simulation and real flight experiments for fixed‐wing UAVs validate the feasibility and reliability of the proposed method. The results show that learning‐based methods achieve comparable accuracy in position estimation, outperforming line‐based and contour‐based methods. The average position errors from flight experiments in the x ‐, y ‐, and z ‐axes are 10.31, 5.46, and 7.37 m, respectively. In contrast, the average error for line‐based and contour‐based methods is significantly higher, at 25.49 m on the x ‐axis.