Generalized Aberrations for Processing-Aware Optical Design

Modern cameras increasingly rely on AI-driven processing, establishing end-to-end (E2E) optimization as a key paradigm in applications ranging from smartphone photography to biomedical imaging. Processing-aware optical design—optimizing the optics for downstream processing—is central to E2E pipelines, yet typically relies on ineffective first-order optimization. Robust pseudo-second-order methods from conventional design, such as Levenberg-Marquardt, generally fail in this context: because such design often relies on a single scalar-valued loss function, it provides insufficient constraints for the high-dimensional optical parameter space, leading to rank-deficient Jacobians that break conventional least-squares solvers. To address this gap, we introduce the generalized transverse ray aberrations (GTRA): a generalization of the well-established TRA formulation underlying industry-standard spot-radius optimization. By lifting scalar task-driven losses into high-dimensional ray-level residuals, the GTRA objective unlocks Levenberg-Marquardt solvers to bring together the robustness of traditional design with the flexibility of E2E pipelines. We validate our method on over 100 design instances for E2E image restoration, including smartphone telephoto lenses, microscope objectives, and C-mount cameras. Our optimization decisively outperforms first-order methods, yielding designs that consistently surpass spot-radius-optimized counterparts in image quality or form factor. These findings demonstrate that the GTRA framework extends the robustness of conventional design to the processing-aware setting.