TMAR: 3-D Transformer Network via Masked Autoencoder Regularization for Hyperspectral Sharpening

Fusion-based hyperspectral super-resolution techniques are utilized to increase the spatial resolution of a hyperspectral image (HSI) by fusing it with a high spatial resolution assistive image. Transformers have shown high efficiency in vision tasks due to their ability to learn global and long-range information. Many networks have utilized them in vision tasks such as super-resolution. However, the employment of convolutional neural networks (CNN) or transformers often results in considerable computational complexity. In addition, prior networks often overlook the regularization term separately, which is an absent factor in these networks. In this study, we focus on leveraging the power of CNN and transformer models and propose a multi-stage deep transformer-based super-resolution network that is regularized via an asymmetric autoencoder structure. In addition, we utilize a 3D convolution layer in the light transformer structure because it allows for more flexible computation of correlations between HSI layers and better capturing of dependencies within spectralspatial features. We apply a spectral masking autoencoder in an asymmetric structure to extract superior prior features from training data and regularize the network. Experimental results on remote sensing HSI datasets demonstrate that our proposed network provides superior efficiency compared to the state-ofthe-art fusion-based super-resolution approaches.