Fusion of Hyperspectral image and multispectral image based on two-branch spectral unmixing network

Image fusion is a mainstream approach to improve the spatial resolution of hyperspectral images (HSI). This paper proposes a fusion method of HSI and multispectral image(MSI) based on a novel two-branch spectral unmixing network which is divided into a fusion branch and a residual branch. Fusion branch firstly extracts endmembers of the input HSI, then fuses the HSI and MSI with a PAN image to obtain high resolution abundances. So the high resolution HSI is acquired by combining endmembers and abundances. In addition, considering the effect of multiple non-linear factors, a residual branch is introduced in this work to complement spectral and spatial details. Finally, the results of fusion branch and residual branches are combined to obtain the more accurate reconstruction HSI. Experiments on both simulated and real datasets show the effectiveness of this work.