PGMNet: A Prototype-Guided Multimodal Network for Ship Recognition in SAR Images

Ship recognition in synthetic aperture radar (SAR) images has extensive applications across various fields. However, the substantial intra-class variability and inter-class similarity present inherent challenges to achieving high-precision recognition. Speckle noise in the background reduces the signal-to-noise ratio, complicating the extraction of discriminative characteristics. Additionally, traditional convolutional neural networks-based methods, which rely solely on image processing frameworks without leveraging additional modality information, struggle to accurately represent the features of SAR targets with bright scatters. To address these issues, we propose a prototype-guided multimodal network (PGMNet), marking a pioneering effort to introduce an image-text multimodal fusion processing paradigm into SAR ship recognition. First, a generation strategy of the prototype and the key area is designed to improve the distinguishability between targets and backgrounds. Besides, a prototype-guided alignment module (PGAM) is implemented to assist the network in characterizing key area information, enhancing intra-class feature consistency. Furthermore, a text feature processing branch is incorporated to precisely describe ship size information and effectively integrate image-text multimodal features, reducing intra-class feature distance while enlarging inter-class feature distance. Extensive experiments on the OpenSARShip and FUSARShip datasets demonstrate that the proposed PGMNet achieves state-of-the-art (SOTA) performance. Notably, the accuracy of PGMNet is at least 11% higher than the current SOTA algorithms on the OpenSARShip-VI dataset.