Multi-Layer Adversarial Detection for Ship Segmentation on NVIDIA Jetson Orin

Autonomous systems, including Unmanned Aircraft Systems (UAS), rely on deep learning models for critical tasks such as object detection and segmentation. However, they are targeted by different threats, among them adversarial attacks, where small perturbations in input images can deceive the model and lead it to mispredict. Adversarial images (AEs) remain a challenge, as they are often similar to the expected inputs, making them difficult to distinguish for human observers as well as some models. This study defines a method for detecting AEs in an autonomous system by leveraging multi-level system monitoring, including image-based, model-based, and hardware-based metrics. Our goal is to identify key indicators of adversarial image manipulation while minimizing experimental bias through controlled evaluation conditions. We generate multiple adversarial samples using both white-box and black-box attack strategies. Experiments are conducted using the Airbus Ship Detection dataset, which comprises high-resolution satellite images suitable for aviation-related applications such as aerial coastal monitoring and maritime surveillance. A ship segmentation model is deployed on an NVIDIA Jetson Orin AGX, and metrics are collected during inference under controlled conditions. Our contribution includes, first, identifying a relevant set of features that effectively distinguish AEs from genuine inputs. Second, we investigate if system profiling - by including model and hardware monitoring - can provide additional robust signals for adversarial detection beyond traditional image-based analysis. We propose a real-time adversarial detection pipeline based on supervised classification over Image, Model-profiling, and hardware metrics. The results reveal that the detection relies mainly on image features while profiling features provide complementary cues for some attacks and hardware (Tegrastats) metrics are noisy and largely uninformative.