Tracking radiometrically accurate synthetic moving ground vehicles from time-offset satellite-image bands

Moving objects contained in the field-of-view of multispectral satellite sensors that capture their band images at different times will appear as “rainbow” streaks of radiation in composite images. Coincidingly, various military and commercial applications rely on or are aided by automatic detection and tracking of moving objects, such as vehicles, from remote sensors. In this study, we present Spectral Segmentation and Kinematic Inference for Target Localization and Surveillance (SSKITLS), a detection and tracking system for moving vehicles against complex backgrounds contained in Planet SuperDove multispectral imagery (visible to near-infrared) in which the bands are offset temporally due to sequential acquisition times (separated on the order of seconds). First, we develop a machine learning segmentation architecture based on U-net. A spatial attention mechanism is added to refine the feature maps generated in the encoder based on context from the decoder, dynamically highlighting areas of the feature map that are most relevant for the task. In addition to segmentation across every band, vehicle velocities are included in a multi-faceted loss function with estimations outputted by the same model. We train and quantitatively evaluate our model on radiometrically accurate synthetic images generated by Spectral Sciences’ Quick Image Display (QUID) containing vehicles and SuperDove-equivalent backgrounds from around the world. Results on assorted vehicle types and paint colors demonstrate segmentation F1 scores up to 0.88, bounding box-based accuracy up to 94%, and vehicle velocity estimation R-squared values as high as 0.86. In this study, we also quantify detection confidence, present preliminary qualitative results on cluttered multi-vehicle scenes, and outline ideas for future capability improvements.