MambaMTL: Progressive Mutual-Guided Mamba Multitask Learning for Hyperspectral Image Pansharpening and Classification

Multi-task learning (MTL) serves as a effective technology to improving both the performance of hyperspectral image (HSI) pansharpening and that of downstream classification tasks. However, most of the existing MTL frameworks overlook the close relationship between the two tasks, resulting in suboptimal performance. To solve the problem, we propose a progressive mutual-guided Mamba MTL framework (MambaMTL), which achieves mutual enhancement between the two tasks by interacting the refined spatial-spectral features and the rich class semantic information generated by each task. Specifically, we design a dual-branch Mamba-Unet subnetwork with class-aware refinement (MTL-CRMNet) for HSI pansharpening. It incorporates parallel multi-head class-aware refinement layers (MCRLs) to enhance object details in the reconstructed HSI by enriching class-specific features with semantic information from the classification network, thereby improving pansharpening performance. Besides, we introduce a Mamba-based pyramid subnetwork with hierarchical guided fusion (MTL-MGFNet) for HSI classification. It utilizes the pansharpened HSI to guide multi-scale feature fusion in a feature pyramid manner, enhancing classification accuracy. Notably, both MTL-MGFNet and MTL-CRMNet share the same multi-scale feature encoder, enabling the classification subnetwork to generate spatial-spectral features rich in texture and semantics. The interaction between tasks significantly enhances their respective performance. The evaluation results conducted on relevant datasets illustrate the superiority of the proposed MambaMTL in simultaneously achieving better HSI classification accuracy and higher reconstructed image quality.