End-to-end automatic lens design with a differentiable diffraction model

The lens design is challenging and time-consuming, requiring tedious human trial and error. Recently, joint design of lens and image processing networks based on differentiable ray tracing techniques has emerged, which provides the possibility to reduce the difficulty of traditional lens design. However, existing joint design pipelines cannot optimize all parameters, including materials and high-order aspheric terms, nor do they use diffraction theory to calculate point spread functions (PSFs) accurately. In this work, we propose a fully automated joint design framework, especially for smartphone telephoto lenses, which starts from optical design indicators, uses Delano diagrams to calculate reasonable optical initial structures, and jointly optimizes the lens system and the image processing network. Considering the diffraction effect, a differentiable PSF calculation method based on the Fresnel-Kirchhoff diffraction model is used for end-to-end joint optimization. This work can reduce the difficulty of the lens design and provide an accurate PSF calculation method considering the diffraction effect for end-to-end joint optimization.