Computational optical imaging: on the convergence of physical and digital layers

Optical imaging has traditionally relied on hardware to fulfill its imaging function, producing output measures that mimic the original objects. Developed separately, digital algorithms enhance or analyze these visual representations, rather than being integral to the imaging process. The emergence of computational optical imaging has blurred the boundary between hardware and algorithm, incorporating computation in silico as an essential step in producing the final image. It provides additional degrees of freedom in system design and enables unconventional capabilities and greater efficiency. This mini-review surveys various perspectives of such interactions between physical and digital layers. It discusses the representative works where dedicated algorithms join the specialized imaging modalities or pipelines to achieve images of unprecedented quality. It also examines the converse scenarios where hardware, such as optical elements and sensors, is engineered to perform image processing, partially or fully replacing computer-based counterparts. Finally, the review highlights the emerging field of end-to-end optimization, where optics and algorithms are co-designed using differentiable models and task-specific loss functions. Together, these advancements provide an overview of the current landscape of computational optical imaging, delineating significant progress while uncovering diverse directions and potential in this rapidly evolving field.