Bimodal and trimodal image fusion: A study of subjective scores and objective measures

Abstract Thermal vision significantly enhances visibility under various environmental conditions. So, this paper presents a comprehensive study on the importance of thermal vision in improving image fusion human visual perception through subjective evaluation. The study focuses on the fusion of three imaging sensors commonly used in computer vision applications: long-wavelength infrared (LWIR), visible (VIS), and near-infrared (NIR). Four image fusion alternatives (LWIR+VIS, LWIR+NIR, NIR+VIS, and LWIR+NIR+VIS) are produced using a reliable deep learning approach and assessed using both subjective tests and objective metrics. The subjective evaluation is performed involving 15 military students and officers from the University of Defence in Belgrade, while objective assessment is elaborated using eight no-reference measures. Results indicate that fused images with thermal information show better visual performance than non-thermal based image fusion alternative (NIR+VIS). Moreover, LWIR+NIR+VIS and LWIR+NIR fused images provide similar visual appearance, demonstrating that the bimodal image fusion (LWIR+NIR) can be sufficient to produce a highly informative fused image. Additionally, the degree of agreement between subjective and objective scores is calculated. The simple edge intensity measure shows the highest degree of agreement, while the image entropy demonstrates the second-best score.