Deep multimodal unmixing of hyperspectral images using Convolutional Block Attention Module (CBAM) and LiDAR features

Hyperspectral image unmixing has garnered considerable attention across various application domains, particularly remote sensing applications. However, relying solely on one modality to distinguish objects with similar spectral information presents several shortcomings. Enhanced performance can be achieved by integrating geographical information from Light Detection and Ranging (LiDAR) data into Unmixing. This paper introduces a new unmixing model that combines hyperspectral and LiDAR data. Impressive data representation and feature extraction using deep learning technology have been employed to develop the Multimodal Hyperspectral Unmixing Model using CBAM (Convolutional Block Attention Module) attention (MHUCBAM). The model exemplifies a sophisticated approach to multimodal unmixing, incorporating Spectral Spatial attention alongside the CBAM. Channel Attention improved the model’s capability to analyze complex spatial and spectral relationships. Our model achieves accurate unmixing of complex environments with effective multimodal data representation and deep feature extraction. Two real-world multimodal unmixing datasets, namely, Houston and Muffle, are used for the performance evaluation. A rigorous ablation analysis was performed to validate the performance of the proposed model. The comparative study with existing unmixing models demonstrated that utilizing latent features from LiDAR data resulted in better unmixing outcomes in terms of both Root Mean Square Error (RMSE) and Spectral Angular Distance (SAD).