HOTNet: A hybrid optimization and transformer network for hyperspectral image super-resolution

Hyperspectral imaging is a powerful tool that captures detailed information across a wide range of light wavelengths. This makes it useful for medical scans, studying the environment, and observing Earth from space. Cost and sensor technology restrictions hinder the obtaining of high-resolution hyperspectral images. Our work aims to solve this problem by improving both the details and spectral information in low-resolution hyperspectral images. This is achieved by enhancing both the image’s sharpness, i.e., spatial resolution, and different wavelengths, i.e., spectral resolution. This is modeled as a model-based energy function through the alternative direction multiplier method, which is then unfolded in a multistage iterative process. The outcome is further augmented by restoration and enhancement by a hyperspectral multi-attention transformer. The model-based approach alone provides satisfactory results but misses enhanced capabilities of the transformers, which can be augmented with several attention mechanisms to cover important features across various dimensions. Therefore, a carefully designed multi-attention transformer is augmented with the above model, making the entire assembly lightweight, efficient, and scalable. Experimental results reveal that this hybrid approach achieves even better quantitative and visual reconstructions, surpassing existing methods without much affecting the computational costs.