A multiscale feature enhancement and adaptive perception network for object detection in remote sensing image

Abstract Remote sensing object detection (RSOD) is vital for environmental monitoring, resource exploration, and intelligent transportation. However, remote sensing scenes face challenges like complex backgrounds, diversity, and large-scale variations. Existing models struggle to detect small objects and those with significant scale changes. Additionally, the high computational complexity of current algorithms limits real-time processing suitability and increases latency and power consumption. In response to these challenges, we developed a novel network called the multiscale feature enhancement and adaptive perception network (MSAP-Net), specifically designed to address the difficulty of detecting objects with significant scale variations at various scales. Firstly, we design a lightweight dynamic awareness enhancement module , which adaptively senses the shape and position of complex objects through dynamic offset operation to speed up model inference. Secondly, a multi-scale feature enhancement module is proposed, which improves the local sensing capability of the backbone network and increases the richness of feature extraction. Finally, a dynamic detection head is used as the detection head of the network, and different sensing mechanisms are used to enhance the feature representation ability of the model to extract information in the prediction process accurately. We validate the effectiveness of MSAP-Net on the publicly available remote sensing datasets object detection in optical remote sensing images (DIOR), dataset for object detection in aerial images (DOTA), and RSOD with detection accuracies of 88.6%, 73.2%, and 98.9% (in terms of <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" overflow="scroll"> <mml:mrow> <mml:msub> <mml:mrow> <mml:mi>mAP</mml:mi> </mml:mrow> <mml:mrow> <mml:mn>50</mml:mn> </mml:mrow> </mml:msub> </mml:mrow> </mml:math> ), respectively, outperforming the suboptimal network by 2.1% on the DIOR dataset in particular. Furthermore, when we deployed it on the Jetson Orin NX CLB edge device and tested it with the unmanned aerial vehicle detection and tracking (UAVDT) video dataset, it achieved an frames per second of 59.9 (in IMAX power mode), proving its utility in resource-constrained application scenarios. The source code is available at: https://github.com/NmUvS/MSAP-Net .