Channel–Spatial Aligned Global Knowledge Distillation for Underwater Acoustic Target Recognition

Knowledge distillation (KD) is a predominant technique to streamline deep-learning-based recognition models for practical underwater deployments. However, existing KD methods for underwater acoustic target recognition face two problems: 1) the knowledge learning paradigm is not very consistent with the characteristics of underwater acoustics and 2) the complexity of acoustic signals in ocean environments leads to different prediction capacities in teacher and student models. This induces feature misalignment in the knowledge transfer, rendering suboptimal results. To address these problems, we propose a new distillation paradigm, i.e., channel–spatial aligned global knowledge distillation (CSGKD). Considering that the channel features (indicating the loudness of signals) and spatial features (indicating the propagation patterns of signals) in Mel spectrograms are discriminative for acoustic signal recognition, we design the knowledge-transferring scheme from “channel–spatial” aspects for effective feature extraction. Furthermore, CSGKD introduces a global multilayer alignment strategy, where all student layers collectively correspond to a single teacher layer. This allows the student model to dissect acoustic signals at a granular level, thereby capturing intricate patterns and nuances. CSGKD achieves a seamless blend of richness and efficiency, ensuring swift processing while being detail oriented. Extensive experiments on two real-world oceanic data sets confirm the superior performance of CSGKD compared to existing KD methods, i.e., achieving an accuracy (ACC) of 82.37% (<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\uparrow$</tex-math></inline-formula> 2.49% versus 79.88%). Notably, CSGKD showcases an 8.87% improvement in the ACC of the lightweight student model.