End-to-end design of the metalens array imaging system with extended depth of field

The introduction of the metalens array brings technological innovation and performance enhancements to light field imaging systems. While end-to-end design methods overcome the limitations imposed by the manual design of metalens array functionalities, exploration of their potential in achieving an extended depth of field remains unexplored, to the best of our knowledge. This paper proposes a framework for the end-to-end design of a metalens array imaging system tailored for extended depth of field imaging tasks, comprising an image formation model, an image reconstruction model and a specialized training algorithm. This framework effectively covers the extended depth range by leveraging depth importance sampling. The proposed image enhancement network utilizes a multi-scale residual convolution network incorporating channel aggregation and channel attention modules to obtain the final reconstruction results. The end-to-end training process is guided by a composite loss function and employs an alternating training algorithm enhanced with a stop-gradient strategy. Experiments conducted under two depth of field settings demonstrate, through both quantitative metrics and qualitative visual evaluations, that the proposed framework achieves superior imaging performance. Specifically, it demonstrates PSNR improvements of 1.42 dB and 10.03 dB, respectively, when different metalens parameterizations are applied, compared to the conventional design in the 11-times depth of field setting. Ablation studies further validate the effectiveness of the individual techniques proposed. To the best of our knowledge, this is the first work to integrate end-to-end optimization with metalens array design for extended depth of field imaging, addressing the limitations of traditional approaches and improving performance over a broad depth range.