Multi-Modal Fusion Detection Method Based on Modal-Specific Dominance Information Propagation Module

Simultaneously using infrared and optical images for target detection is becoming a research hotspot. The technology of using multi-modal data for high-performance target detection has proved its great competitiveness in practical applications. However, infrared images and optical images have different emphasis on dominant information, and direct fusion or symmetric feature processing may reduce the learning efficiency of the network, thus affecting the detection performance. To solve these problems, a new fusion detection method based on modal-specific information propagation (MSIP) is proposed in this paper. It mainly includes two innovations: a spatial information propagation module (SaIDM) is designed to efficiently propagate a large amount of spatial information in optical images to the infrared branch. At the same time, a semantic information propagation module (SeIDM) is designed, which uses the semantic information in the infrared branch to enhance the attention of the optical branch. The experimental results on the DroneVehicle dataset show that the network can improve the detection accuracy of difficult targets by transmitting modal dominance information, and the mAP reaches 71.13, which is in the leading level.