YOLO-Shuffle Unified Cross Attentive Feature Fusion for Multispectral Drone Based Object Detection

Drones have become increasingly important in applications like surveillance, benefiting from advancements in deep learning and sensors. While RGB cameras have traditionally been used for their ability to capture detailed features, the integration of infrared (IR) cameras enhances detection, especially in challenging conditions. However, combining RGB and IR data introduces challenges such as increased model complexity, redundancy, and misalignment, which can degrade performance. To address these issues, transformer-based models have been used to manage long-range dependencies and mitigate misalignment. Despite their effectiveness, these models can be complex and require further refinement for better efficiency. A key component for improving detection performance is the backbone structure, which benefits from efficient fusion mechanisms. This study proposes a new model with a developed shuffle-based backbone (YOLO-Shuffle) and Unified Cross Attentive Fusion (UCAF), cross-transformer-based fusion module. It has been tested on three diverse datasets; DroneVehicle for vehicles detection using drones, KAIST for pedestrian detection, and FLIR, which includes three object classes. The importance of this study lies in its potential to improve detection accuracy and efficiency in drone-based surveillance systems, especially in complex, real-world environments. Results demonstrate outperforming performance for the DroneVehicle dataset with mAP of 78% with an increase of 1.8% with the YOLO-Shuffle backbone and the base model fusion with the reduction in model size by 32.8% compared to the related state of the art studies, achieving a trade-off between speed and accuracy with more efficient performance. The base model size is reduced by 26% for the YOLO-Shuffle backbone with ICAFusion, and by 44.3% for the YOLO-Shuffle backbone with the UCAF fusion. Speed is increased by 14.5% and 51% for the KAIST and FLIR datasets respectively, while maintaining accuracy across FLIR, and increase by 1.4% for the KAIST datasets.