End-to-end snapshot metalens spectral imaging based on deep unfolding half-shuffle transformer network

End-to-end computational spectral imaging based on metalens, which enables compact spectral imaging systems by co-optimization of optical encoder and computational decoder, has attracted extensive attention. However, traditional end-to-end spectral imaging systems typically employ pure data-driven convolutional neural networks (CNN) that ignore physical priors of the imaging process and long-range dependencies, limiting high-precision spectral reconstruction. In this paper, we propose and experimentally demonstrate an end-to-end snapshot metalens spectral imaging based on a physics-driven deep unfolding half-shuffle Transformer network. By unfolding the iterative process of traditional optimization algorithms into a cascaded network structure, we not only integrate the advantages of adaptive feature learning neural network and physical model, but also introduce the physical priors of optical-encoder into the network to allow physically-consistent reconstruction process. Besides, a multi-head self-attention mechanism half-shuffle Transformer network is utilized to extract both local and global information. In this way, we achieve stronger generalization and higher reconstruction quality compared with traditional CNN-based reconstruction methods. We designed and fabricated the metalens, achieving high-fidelity spectral image reconstruction across 25 channels (430-670 nm) at a spatial resolution of 512 × 512 with an average peak signal-to-noise ratio (PSNR) of 42.87 dB, while preserving rich spatial details compared with state-of-the-art (SOTA) methods. Our proposed framework facilitates full interaction between the encoder and the decoder, boosting the power of joint optimization in end-to-end design, and paves a way for the design of miniaturized spectral imaging metalenes.