Multi-Layer Adversarial Detection for Ship Segmentation on NVIDIA Jetson Orin

2025
Deutscher Luft- und Raumfahrtkongress (DLRK)

Authors: Badr-Eddine Bouhlal and Clemens-Alexander Brust

Abstract: Autonomous systems, including Unmanned Aircraft Systems (UAS), rely on deep learning models for critical tasks such as object detection and segmentation. However, they are targeted by different threats, among them adversarial attacks, where small perturbations in input images can deceive the model and lead it to mispredict. Detecting these adversarial images remains a challenge, as they are often similar to the expected inputs, making them difficult to distinguish for human observers as well as some models.

This study defines a method for detecting adversarial images in an autonomous system by leveraging multi-level system monitoring, including image-based, model-based, and hardware-based metrics. Our goal is to identify key indicators that signal adversarial manipulation of an image while reducing the bias by controlling the experimental conditions.

We generate multiple adversarial samples using both white-box and black-box attack strategies. Experiments are conducted using the Airbus Ship Detection dataset, which comprises high-resolution satellite images suitable for aviation-related applications such as aerial coastal monitoring and maritime surveillance. A ship segmentation model is deployed on an NVIDIA Jetson Orin AGX, and metrics are collected during inference under controlled conditions.

Our contribution includes, first, identifying a relevant set of features that effectively distinguish adversarial images from genuine inputs. Second, we investigate if system profiling - by including model and hardware monitoring - can provide additional robust signals for adversarial detection beyond traditional image-based analysis. We also propose a detection method that integrates the selected metrics for real-time adversarial image detection. The results reveal that the detection relies mainly on image features while profiling features provide complementary cues for some attacks and hardware (Tegrastats) metrics are noisy and largely uninformative.