TwinFusion-YOLOv9 for RGB-T Object Detection

Cooperative target detection using infrared and visible images is currently a highly popular approach. This approach is not affected by environmental lighting conditions and can operate around the clock. However, issues such as inaccurate time and spatial synchronization of sensors, target movement, stereo disparity, or other factors leading to positional deviations result in the same target not overlapping completely in different images. This can impact the overall detection performance of the model. To tackle these challenges, this paper introduces a target detection network named TwinFusion-YOLOv9, which integrates features from visible images and infrared images to enhance detection robustness. To ensure the efficiency and accuracy of the solution, a structure suitable for joint target detection based on YOLOv9 was designed. Features from visible and infrared images are separately extracted using a backbone, then fused using a newly designed fusion module, and detection results are obtained using downstream modules. During the training process, an auxiliary detection branch improved from Programmable Gradient Information (PGI) proposed in YOLOv9 is incorporated to enhance model accuracy and expedite convergence. Experimental results demonstrate that TwinFusion-YOLOv9 exhibits outstanding performance on public datasets, holding both theoretical and practical significance.