<i>MVMFCam</i>: an all-in-focus optical synthetic aperture imaging system based on multi-view multi-focus computational imaging

Due to the properties of optical lenses, usual imaging devices suffer from a limited depth of field (DoF), and objects outside this area are blurred. To overcome the limited DoF, a common method is to continuously adjust the focal length or focal plane of the imaging system to capture a set of multi-focus images, and then fuse them into an all-in-focus image. However, such imaging mechanisms cannot capture multi-focus images simultaneously, thus failing to achieve all-in-focus imaging for each frame in dynamic scenes. In this paper, to overcome this limitation, we propose a novel all-in-focus optical synthetic aperture imaging system (<i>MVMFCam</i>) based on multi-view multi-focus computational imaging. <i>MVMFCam</i> is a camera array composed of nine sub-cameras, where each sub-camera focuses at different depths of the scene according to specific focusing rules to capture all clear details of the scene. <i>MVMFCam</i> is controlled by a synchronous clock module, which can capture multi-view multi-focus (MVMF) images simultaneously in a single exposure. For MVMF image fusion, we further propose an end-to-end MVMF image fusion neural network (MVMF-Net). MVMF-Net consists of two phases: image alignment based on a feature transfer matching strategy and adaptive fusion based on a densely connected network. Firstly, the MVMF images are aligned into a set of focal stack images in the spatial coordinate system of the selected reference view. Subsequently, the focal stack images are input into a densely connected fusion network based on adaptive fusion weights for fusion to obtain the final all-in-focus result. In order to verify the all-in-focus imaging capability of <i>MVMFCam</i>, we capture 10 sets of MVMF testing datasets. The experimental results show that <i>MVMFCam</i> can achieve high-quality all-in-focus imaging for each frame in dynamic scenes, which will be beneficial to the development of high-performance computational imaging technologies and devices, as well as their applications in key fields such as microscopic imaging, close-range photography, and non-destructive testing.