Vehicle Target Detection Based on Cross‐Modality Projective‐Invariant Features Extracted from Unpaired SAR and Infrared Images

ABSTRACT Synthetic aperture radar (SAR) automatic target recognition (ATR) is remarkably challenging since the SAR image defies the foundation for human and computer vision, i.e., the Gestalt perceptual principles. We propose to address this problem by fusing the target features reflected in SAR and infrared (IR) images via a novel dual‐channel context‐guided feature‐alignment network (CGFAN) that is capable of fusing the cross‐modality projective‐invariant features extracted from unpaired SAR and IR images. First, region of interest (ROI) matching between SAR and IR images is realized based on special landmarks exhibiting consistent cross‐modality features. After that, generative models trained with historical SAR and IR images are used to synthesize SAR images based on the IR images collected in real time for the current mission. Since SAR imaging takes more time than IR imaging, by using these synthesized SAR images as auxiliary data, the spatial‐coverage rate in a typical collaborative SAR/IR ATR mission carried out by drone swarms is effectively improved. The proposed CGFAN is tested against the proprietary monostatic‐bistatic circular SAR and IR dataset constructed by the researchers at our institution, which consists of nine types of military vehicles. Experimental results show that the proposed CGFAN offers better ATR performance than the baseline networks.A novel dual‐channel CGFAN that is capable of fusing the cross‐modality projective‐invariant features extracted from unpaired SAR and IR images is proposed. First, ROI matching between SAR and IR images are realized based on special landmarks exhibiting consistent cross‐modality features. After that, generative models trained with historical SAR and IR images are used to synthesize SAR images based on the IR images collected in real time for the current mission.