A novel mosaic adaptive attention network for multispectral imaging reconstruction

Due to the sparse sampling of each spectral band in MSFA, the more spectral bands, the lower the spatial resolution of each frequency band. It is still a challenge to obtain high-resolution snapshot images in both spectral and spatial domains in a short exposure. In addition, the resolution of existing narrow band MSFA-based imaging systems in the spectral dimension is limited by the number of filters. The number of filters is equal to the number of bands in a multispectral image. To overcome these difficulties, we propose a novel multispectral imaging system. By sampling the spectra with random broadband filter-based MSFA, more spectral information is preserved in the generated spectral mosaic images. We further present an end-to-end mosaic adaptive attention network (MAAN) for multispectral image reconstruction. It contains a mosaic convolution block (MCB), four residual dense attention blocks (RDAB), and two spatial attention blocks (SAB). The MCB is used to extract mosaic features based on mosaic patterns. The RDAB is designed to learn deeper mosaic features, where channel attention block (CAB) can highlight the useful channel feature information. The SAB can enhance the useful spatial structure features. The experimental results show that the proposed MAAN algorithm outperforms existing CNN-based multispectral image reconstruction algorithms.