Detecting electro-optical sensor anomalies in maritime satellite imagery using classification model latent space embeddings

Artificial Intelligence (AI) based computer vision algorithms have the ability to address key challenges in detecting naval objects of interest in commercial, satellite Electro Optical (EO) imagery; however, reaching high reliability in operations can be challenging. For naval applications, training data may be incomplete, and sensor data may have increasing noise as sensors age. As such, it is necessary to create models that can detect when the computer vision models are operating outside of the domain of their training data, so that end-users can disregard what may be unreliable model conclusions. Models are developed to estimate whether input imagery has been corrupted by sensor anomalies, such as excess sensor noise, excess dead pixels or focus problems. These models are based on detecting features in the image classification model’s latent feature space. While the highest performing model was a ResNet model trained with the IsoMax+ loss function, two of the models that made use of the latent feature vector information also performed well. Algorithms that were trained using anomalous image examples outperformed models that were not trained with anomalous examples. Future work should examine how best to make use of these various methods to increase model robustness, which is often needed for maritime applications, where safety can be critical.