Three-Decoder Cross-Modal Interaction Network for Unregistered RGB-T Salient Object Detection

Substantial advancements have been made in the field of salient object detection in image processing in recent years. This research introduces the three-decoder cross-modal interaction network (TCINet) for salient object detection in unregistered red-green-blue (RGB)–thermal image pairs, modeling information from different modal perspectives. TCINet employs a three-decoder framework to process RGB, thermal, and fused feature maps concurrently. To ensure robust integration between the modalities, mitigating the impact of unregistered images and addressing modality imbalances, we introduce the fusion complementary registration (FCR) module. This module guides attention to connect the two modalities and uses atrous spatial pyramid pooling (ASPP) to adapt to image scale changes. To fully utilize the differences between modalities, we designed two distinct decoders: fusion feature decoder (FFD) for decoding the fused features and single-modal decoder (SMD) for decoding single-modal features. Additionally, we incorporated feature enhancement (FE) units into the modal decoding to mitigate the blurring effect caused by high-speed autonomous aerial vehicle (AAV) flight. We use a weighted fusion module (WFM) to dynamically integrate the features decoded by the three decoders to increase the network’s generalization ability. Extensive experiments show that TCINet outperforms existing methods, achieving excellent results on a variety of challenging scenarios containing complex details. The code will be published at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/zqiuqiu235/TCINet.git</uri>.