Towards Multi-Class Open-Set Recognition by Use of Lower Dimensional Latent Space Embeddings

An increasing number of space-based optical imaging systems are deployed to orbit. While these systems have the potential to address long-standing challenges in maintaining maritime security by using deep learning-based imagery classification to detect spurious maritime activity, such as illegal fishing, there exists additional challenges to overcome. Namely, deep-learning based classifiers will need to identify imagery that is outside the representation of the training imagery, known as open-set imagery, such that maritime operations can effectively accommodate scenarios where new objects, such as new vessels, are imaged. This paper extends previous work by developing a new open-set detection method based on the Parametric T-distributed Neighborhood Embedding (PTSNE). The PTSNE method is used to train a lower dimensional model that maps an image's latent feature vector from the native, high-dimensional space, to a lower dimensional space based on the underlying training data. From there, a linear Support Vector Classifier (SVC) model is used to capture the decision boundary in the lower dimensional embedding between each object in a one-vs-rest fashion. The combination of the PTSNE and SVC models are referred to as a Lower Dimensional Embedding Classifier (LDEC). Imagery that has a high predicted probability output from the SVC would correspond to images that more closely match the lower dimensional embedding of the training data for that object, whereas imagery with a lower predicted probability output from the SVC would correspond to images outside the typically training data for the object, and thus may be unknown imagery. This behavior is leveraged to identify novel classes of data. Two experimental cases are constructed using the CIFAR10 dataset and a commercial maritime electro-optical dataset respectively. The LDEC model is compared to the Openmax model, as well as the Binary Classifier (BC) and Category-aware Binary Classifier (CBC) models which identify open-set data based on latent space representations. In both cases, the LDEC model outperforms the other models examined. For the maritime test case, The LDEC model shows an 11% increase in the Area Under the ROC curve compared to the next highest performing model, though this difference is closer in the CIFAR case. Overall, the technique of leveraging lower dimensional embeddings to identify class outliers in a representation of a baseline model's latent improves on existing methods.