Spatial–spectral fusion remote sensing detection with channel-spatial attention and context augmentation and a high-quality benchmark

Remote sensing object detection relying solely on a single data source, such as panchromatic images (single-band grayscale images that cover a broad spectral range) or multispectral images (images synthesized from multiple narrow spectral bands), often suffers from suboptimal detection accuracy. This is mainly attributed to the inferior quality of the images. Addressing this, we introduce an end-to-end fusion and detection integration network (FADINet) that jointly utilizes both high spatial resolution panchromatic images and multispectral images. The FADINet aims to enhance the detection performance without any specific preprocessing step. Specifically, by feeding both types of images into the network and utilizing the channel-spatial attention feature fusion module, information from different bands can be adequately integrated. Furthermore, to solve the problem that tiny objects in remote sensing images are difficult to detect due to low resolution and small size, we propose a context augmentation module (CAM). The CAM injects the features of multiscale dilated convolution (convolution operation capturing multiscale features by controlling kernel dilation rates) into the feature pyramid networks from top to bottom to supplement the context information. In addition, to address some of the shortcomings of the existing datasets, we created a high-quality dataset called SHAPL, comprising numerous instances of both aircraft and ships. Comprehensive experiments on both created and public datasets demonstrate the effectiveness of the proposed approach compared with the state-of-the-art methods. Specifically, the method proposed in this paper achieves 89.35% mAP on the DOTAv1.0 dataset (only for the categories of aircraft and ships), 90.50% mAP on the SHAPL dataset, and 94.09% mAP on the HRSC2016 dataset.