Drone-based bistatic SAR-infrared cross-modality vehicle detection and classification

Unlike computer vision datasets, aerial images often come with prior knowledge on the distance between the object of interest and the sensors, which makes it possible to recover the sizes of the targets before making any predictions. On the other hand, these images usually consist of large amount of both out-of-library (OOL) targets of high values and clutters, which makes it very challenging to strike a balance between recall rate and precision. To effectively combine the complementary information contained in the aerial images collected by the synthetic aperture radar (SAR) systems and infrared (IR) sensors mounted on drones, a novel two-stage cross-modality vehicle detection and classification strategy is proposed. In Stage 1, all the potential targets are sorted out to form a large candidate pool by jointly considering the target features exhibited in both the SAR and the IR images so that targets with low radar cross section (RCS) are preserved. In Stage 2, multichannel convolution neural network (CNN) models are used to identify the in-library targets based on the multiview target chips while rejecting all the others as either OOL objects of interest or clutters. Experiment results show that the proposed two-stage multiview cross-modality network model architecture offers superior target detection and classification performance than the single-stage deep neural networks relying solely on the IR or SAR data.