Deep learning for unambiguous pose estimation of a non-cooperative fixed-wing UAV

Abstract Image-based unmanned aerial vehicle (UAV) pose estimation is carried out by deep learning (DL) to automate the UAV aim-point selection for a laser beam control research testbed (LBCRT). DL models are proposed to estimate the UAV pose from available UAV target images and serve as a new sensor within the testbed. Such models are designed to address the problem of pose ambiguity, which arises from different 3D UAV poses looking similar in a 2D image plane. The models are trained under three datasets: synthetic data, synthetic data combined with real-world data, and real-world data. Afterward, the models are compared based on their ability to infer a real-world UAV trajectory in order to analyze the impact of using synthetic data during training. Quantitative results show that the proposed DL models solve the pose ambiguity problem when trained with appropriate data. In average, the models trained on real-world data had the lowest mean angle error, followed by those trained on combined data, and those trained solely on synthetic data. This shows that synthetic data should be carefully selected to bridge the gap when real-world data is unavailable or scarce. Customized real-world and synthetic data were created for the research. Real-world data are short wave infra-red (SWIR) images of a 3D printed UAV model with varying poses and associated pose labels. In addition, synthetic data are UAV images with varying poses and associated labels created by simulation. For this research, 77,077 real-world and 100,000 synthetic data were created.