Novelty Detection in Underwater Acoustic Environments for Maritime Surveillance Using an Out-of-Distribution Detector for Neural Networks

Reliable detection of unknown signals is essential for ensuring the robustness of underwater acoustic sensing systems, particularly in maritime security and autonomous navigation. However, Conventional deep learning models often exhibit overconfidence when encountering unknown signals and are unable to quantify predictive uncertainty due to their deterministic inference process. To address these limitations, this study proposes a novelty detection framework that integrates an out-of-distribution detector for neural networks (ODIN) with Monte Carlo (MC) dropout. ODIN mitigates model overconfidence and enhances the separability between known and unknown signals through softmax probability calibration, while MC dropout introduces stochasticity via multiple forward passes to estimate predictive uncertainty-an element critical for stable sensing in real-world underwater environments. The resulting probabilistic outputs are modeled using Gaussian mixture models fitted to ODIN-calibrated softmax distributions of known classes. The Kullback-Leibler divergence is then employed to quantify deviations of test samples from known class behavior. Experimental evaluations on the DeepShip dataset demonstrate that the proposed method achieves, on average, a 9.5% and 5.39% increase in area under the receiver operating characteristic curve, and a 7.82% and 2.63% reduction in false positive rate at 95% true positive rate, compared to the MC dropout and ODIN baseline, respectively. These results confirm that integrating stochastic inference with ODIN significantly enhances the stability and reliability of novelty detection in underwater acoustic environments.