Monocular lensless depth camera driven by end-to-end deep learning with a synthetic dataset

We propose a compact monocular lensless depth camera driven by an end-to-end deep learning approach using a synthetic RGB-D dataset. Our lensless depth camera is composed of a fabricated ultra-thin phase mask with a designed point-spread function (PSF) and an off-the-shelf CMOS image sensor. Compared to typical optimization-based reconstruction in lensless imaging, data-driven methods can reconstruct high-quality images using supervised learning. Because there is no existing real-world dataset for lensless depth cameras, a synthetic dataset is needed. In this work, we present a lensless depth estimation framework using synthetic dataset generated within the virtual environment of Blender, a 3D modeling software. By leveraging the rendered RGB and depth maps from Blender and the measured 3D PSF of our device, we can generate a dataset of simulated raw measurements for the lensless depth camera. We propose 3D Multi-Wiener Deconvolution Network trained on our synthetic dataset and report on depth estimation performance quantitatively on simulated test dataset (PSNR > 21 dB, SSIM > 0.93, RMSE < 0.1) and qualitatively on a real-world dataset.