MSFD: Multiscale Feature Decomposition for Cross-Modality Visible-to-Infrared Drone Image Translation

In the global landscape of the Internet of Things (IoT), drone IoT technology has gained widespread application. This technology can monitor and analyze land use and land cover more quickly and more accurately. Currently, the images collected by drone IoT technology are mostly visible images, which are highly susceptible to external environmental factors, while the acquisition of infrared images is relatively more challenging. Visible-to-infrared drone image translation seeks to convert visible drone images into their corresponding infrared counterparts. Although existing GAN-based image-to-image translation methods have demonstrated impressive results in the domain of natural images, they still face challenges in generating highly realistic infrared drone images. Therefore, a novel Multi-Scale Feature Decomposition (MSFD) method is introduced for visible-to-infrared drone image translation. The proposed approach accomplishes the translation through spectral feature disentanglement and cross-modal recombination. In our model, spectral feature disentanglement is based on the separation of modality-specific spectral information and modality-invariant shared structural content from the image representation. Subsequently, the spectral features and underlying content from different modalities can be recombined by generators to facilitate cross-modality image translation. To enhance the quality of generated images, our method integrates a multi-scale spectral feature encoder to address significant spectral discrepancies between targets and backgrounds in drone images by extracting and fusing spectral features at different scales. Additionally, the strategy of multi-scale generators and discriminators further enhances the generation quality of infrared drone images. The experimental results highlight the superior performance of our model in visible-to-infrared drone image translation.