Multiscale Semantically Modulated Mixed Convolutional Networks for Subpixel Mapping

Due to the limitations of imaging environment and hardware conditions, mixed pixels are common in hyperspectral images, which seriously affects the accuracy of land use coverage mapping. Subpixel mapping (SPM) decomposes mixed pixels to obtain the spatial distribution information of local object components inside the pixel, thereby breaking through the limitations of traditional pixel-level classification and achieving more accurate land use interpretation and refined mapping. Recently, deep convolutional neural networks have demonstrated their potential and effectiveness in SPM. However, in the SPM process, the multiscale spatial context information are not fully utilized in the process of using semantic information for network modulation, and the spatial representation at a more abstract level cannot be fully obtained. Therefore, in response to the above problems, this article proposes a multiscale semantic modulation hybrid convolutional network for SPM. The network obtains multiscale semantic information in semantics by constructing a multiscale semantic modulation module (MSSM) to modulate the backbone network and fully mine the spatial context information. Simultaneously, a hybrid convolutional module integrating, 2-D convolutional neural networks, 3D convolutional neural networks, and attention mechanisms is designed. This module captures joint spatial–spectral features while reducing model complexity and learns more abstract spatial representations to enhance the network’s performance in SPM. Experimental results show that this method outperforms the most advanced SPM methods on three public datasets and a produced wetland dataset, and the details of land cover categories are more prominent. The code and data will be released on GitHub upon acceptance: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/UPCGIT/MSMCNet</uri>