Precise Ranging to an Aerial Refueling Coupler Using a DNN and a Monocular Camera System

The Office of Naval Research's Advanced Au-tonomous Air-to-Air Refueling System project explores the application of deep neural networks (DNN) for automated uncrewed aerial vehicle refueling. In this study, we present a monocular camera system integrated with a DNN to accurately estimate coordinates and range to a refueling drogue within the final 25 feet of approach. Our method employs a similar triangle algorithm that computes the range from DNN-generated bounding boxes, with ground truth provided by a calibrated motion capture system. Experiments using UR10, YASKAWA, and Linear Track manipulators demonstrate that the DNN achieves perfect precision and recall, with a mAP50 of 0.995 and mAP50-95 scores of 0.945 for the drogue, 0.851 for the coupler, and 0.898 overall. Combined with the monocular vision system, the estimated coupler range is within 4 inches of the motion capture measurements for distances between 7 and 25 feet, aside from minor deviations at 20 and 23 feet. This work advances the prospects of automated air-to-air refueling by providing a robust, vision-based solution for accurate target detection and range estimation.