Vision Based Runway Identification with marked runway for Automatic Landing Applications of UAV with onboard monocular and stereo vision Sensing

Vision based runway identification using standard ‘marked runway’ images captured from a fixed wing unmanned aerial vehicle (UAV) through onboard monocular or stereovision sensor is presented in this paper. An innovative convolutional neural netwok (CNN) based ‘You Only Look Once’ (YOLO) object detection algorithm has been exploited to detect the runway threshold region during approach segment of landing of a UAV. This threshold region is defined as Region of Interest (ROI). ORB-SLAM3 algorithm with visual odometry has been configured for feature detection of runway and feature correspondence between successive camera frame’s ‘feature points’ to compute rotational and translational matrices and compute ‘position and orientation’ (POSE) of camera in ORB-SLAM coordinate frame. Perspective n point (PnP) algorithm has been configured for camera pose estimation in runway coordinate frame. Feature detector based image processing algorithm with monocular (with scaling constraint) and stereo vision constraint is used to establish the relation between 3D corner points of ROI and corresponding detected corner feature points. Based on the detected runway, a runway coordinate frame is constructed at ROI center and UAV onboard camera pose in runway coordinate frame is computed in real time.