Dual-Task Cascaded Network for Spatial–Temporal–Spectral Remote Sensing Image Fusion

Spatial-temporal-spectral fusion is dedicated to integrating the complementary advantages of multisource images to obtain fused image with all high spatial, high temporal and high spectral resolutions, which is promising but more challenging. On the one hand, traditional studies deployed on MODIS and Landsat data cannot be transferred to most spaceborne hyperspectral (HS) data with lower temporal resolution; on the other hand, the rigid time relation modeling in most existing studies exhibits weakness orienting to non-linear land-cover changes. In this paper, we propose a dual-task cascaded network for spatial-temporal-spectral fusion, with collaborative modeling on spatialspectral joint enhancement and temporal variation estimation in a unified framework. The spatial-spectral joint enhancement task was designed with an iterative alternating projection, meticulously crafted to address the scale variance among observation. Additionally, the spatial enhancement unit and error correction unit were coupled modeling to enhance the spatial and spectral fidelity. The temporal variation estimation on spectral fine tuning network was developed, to further enhance the temporal and spectral fidelity. Extensive experiments were implemented on Ziyuan(ZY)-1 02D HS data and Sentinel-2 multispectral (MS) data. Both qualitative and quantitative results demonstrated the competitive performance of the proposed method.