To fuse or not to fuse: enhancing military operation object detection with multimodal late fusion and color space optimization

Abstract Object detection in military operations faces critical challenges, including camouflaged targets and occlusion, where traditional RGB-based systems often fail. This study proposes a systematic framework for optimizing multimodal late fusion in object detection by integrating color space transformations with depth information. We contribute three key elements: (1) the development of the “militar-VALID” dataset, a specialized, defense-oriented collection of 6,054 images curated for challenging detection scenarios; (2) a comprehensive statistical evaluation framework comparing four late-fusion algorithms across 247 unique configurations; and (3) the hyperparameter optimization of the optimal fusion configuration through Bayesian search. Leveraging the YOLOv8-small architecture trained on eight parallel color representations (RGB, BGR, Grayscale, HSV, CIELab, YUV, YCrCb, and Depth), we establish that Weighted Boxes Fusion combining RGB, Depth, and HSV modalities delivers statistically significant improvements ( $$p \le 0.001$$ , Cliff’s $$\delta \ge 0.8$$ ). Specifically, this configuration achieves a 1.30% increase in mean Average Precision (mAP@50-95), a 3.20% improvement in precision, and a 3.22% enhancement in Average Precision (AP@50) for small objects compared to RGB-only baselines. Statistical analysis highlights the depth maps as the most impactful modality and HSV as the optimal color space complement. This work provides a quantitative framework for multimodal color space fusion in military object detection and discusses its implications for deployment in high-stakes operational environments.