Runway Detection using a Modified DeeplabV3+ Segmentation Neural Network for Space Applications

Using Vitis AI, a Xilinx framework for executing machine learning models on heterogeneous computing hardware, this work benchmarked the capability of a Xilinx Versal AI Core system-on-a-chip (SoC) to execute a runway detection segmentation neural network. This demonstration was performed to support an NRL developed algorithm for performing landmark based orbit-determination. The radiation tolerant Xilinx Versal Core AI XQR provides an option for high-throughput on-orbit processing to support applications such as neural networks for electro-optical sensor processing. A Xilinx VCK190 evaluation kit was used to perform the benchmarking. The model used in this effort was a modified DeeplabV3+ segmentation model that performed well using a limited data set of 160 training images and 40 test images. The DeeplabV3+ model was altered to allow for a Xilinx Deep Processing Unit (DPU) instantiation of the model that did not rely on custom implementation of any layers in the network. This was done to increase throughput and ease the implementation of the model on the Versal. The data set was built from imagery captured by the WorldView-2, WorldView-3, and GeoEye-1 satellites. In evaluation of the accuracy, the mean-intersection-over-union (mIoU) was determined to be roughly 0.6772 on the floating-point model following training. The model was then quantized from floating point to INT8 in order to allow for FPGA compatibility, and then compiled for execution using Vitis AI on the Xilinx Versal VC1902 SoC hosted on the VCK190 development board. The resulting model was run using the Vitis-AI API via a python script, and the resulting mIoU was found to be 0.6757. The maximum throughput achieved in this configuration was 70.108 FPS using 7 threads for evaluation on the VCK190.