RGB-infrared image fusion and classification based on tensor decomposition

The development of multi-sensor integrated stereo observation technologies from spaceborne, airborne, and terrestrial platforms has brought a rich data source. Consequently, remote sensing images possess multimodal characteristics. Harnessing this richness, the integration of complementary information from multimodal data has facilitated a more comprehensive understanding of the Earth's surface. Successful applications of deep learning techniques, sparse representation methods, and multi-scale, multi-feature decomposition approaches have been observed in multimodal image fusion tasks. Tensor, as a higher-dimensional generalization of vectors or matrices, offers greater flexibility in data analysis techniques through tensorization. Tensor decompositions are capable of extracting more general latent components compared to vector- or matrix-based methods, aligning with various data properties and constraints. Recent advancements have revealed the significant potential of tensor decomposition in hyperspectral and multispectral image fusion, hyperspectral remote sensing image classification, dimensionality reduction, target detection, spectral unmixing, and various other applications. However, research on tensor factorization has seldom explored red-green-blue (RGB) and infrared (IR) data fusion and classification. In this paper, we introduce a novel approach for multimodal image fusion and classification that leverages tensor decomposition. Our method utilizes tensor decomposition to extract underlying correlation information between RGB and IR images of corresponding objects, leading to a substantial enhancement in classification accuracy. Comparative experimental results on three groups of similar objects from the VEDAI dataset demonstrate the superior classification accuracy achieved by our proposed method.