Synthetic Data Generation for Computer Vision Based Autonomous Landing for Small UAS Package Delivery

Small unmanned aerial systems (UAS) are showing great promise for commercial package delivery. Zipline and Google Wing have developed and are operating small UAS fleets for delivery in the United States and internationally. Deep learning based computer vision model on UAS platforms for recognition of the objects near landing zone remains an open area of research, such as pedestrians, pets, cars, etc. Deep learning, neural network based computer vision systems supporting autonomous landings require large amounts of data to train and validate the models. Flight tests are often conducted for video data collection of specific landing or delivery scenarios of interest. However, running real-world flight test experiments are often laborious and costly, not to mention possible regulatory constraints depending on the locations of flight test experiments. In addition, it is extremely challenging to use flight tests to capture a wide of range of landing scenes under different seasonal, atmospheric and lighting conditions, such as evening or low-light, fog, light rain, air pollution, etc. Such challenges motivated us to investigate synthetic data generation using high-fidelity simulation frameworks. This paper reports our approach for modeling realistic landing zone environments using Microsoft AirSim, an open-source Unreal Engine-based simulation platform popular for artificial intelligence (AI) research and experimentation. Also reported herein is our process for scenario specification and generation of realistic dynamic and/or static landing scenes with tunable parameters including weather effects, time-of-day lighting, and building/environmental geometry with respect to the landing area. The synthetic flight data is used for YOLOv8 model evaluation to show that the vision model can make accurate obstacle detections in a simulated landing scenario and the standard mean average precision (mAP) metrics are reported for a synthetically generated flight test dataset.