An Unsupervised Spatial-Temporal-Spectral Fusion Model for Hyperspectral Images Using a Global Shared Convolutional Neural Network

Hyperspectral images (HSI) are renowned for their high spectral resolution and extensive wavelength coverage, but they often suffer from limited spatial and temporal resolution due to imaging sensor constraints. This may make it difficult for hyperspectral images to play a role in the acquisition of fine surface information and the observation of continuity of time scales. Spatial-temporal-spectral fusion (STSF) has the purpose of synthesizing different information which from multisource satellite data which has the advantage in temporal, spatial, and spectral aspect respectively to reconstruct HSI with high spatial resolution and high temporal resolution. However most existing spatial-temporal-spectral fusion methods are restricted to the assumption of linear relationship of space temporal and spectrum. Furthermore, the spatial-temporal-spectral fusion methods mostly cannot directly process the current hyperspectral data which has a lower temporal resolution based on Landsat and MODIS. This paper proposed an unsupervised STSF model for hyperspectral images using a global shared convolutional neural network (UGSCNN). The proposed method has two models: 1) Spatial-spectral down model: combine spectral linear theory with deep learning to down sample the space and the spectrum; 2) Spectral up model: utilize the shared global information to up sample the spectrum. To verify the proposed method, we compared it with others using simulated and real on-board data, proving its effectiveness and practical fusion results.