NUNSRN: Non-uniform and noise sensing restoration network for single-lens imaging

• A spatially varying deblurring method based on nonlinear spatial-variation prior of single lens point spread function. • A denoising method based on contour difference characteristics of optical information and noise. • The mutual influence and the decoupling method of deblurring and denoising. It has attractive benefits by using a single lens to replace the complex stacks of lens elements in imaging system. But the single lens has spatially varying aberrations that can result in non-uniform blurring. Moreover, the modulation transfer function (MTF) of single lens is very low, resulting in significant reduction of the signal-to-noise ratio (SNR). To improve the imaging quality of single lens, a lot of image restoration methods based on deblurring and denoising have been proposed. However, we find that the denoising and non-uniform deblurring affect each other during restoration, which causes artifacts in recovery image. To solve these difficulties in single lens imaging, we propose the non-uniform and noise sensing restoration network (NUNSRN). First, to remove the spatially varying aberration blur, NUNSRN proposes the non-uniform convolutional layer based on pixel-wise kernel estimation with sinc function. Then, NUNSRN proposes the noise sensing structure based on contour prior to distinguish noise and signal at low SNR. Most of all, NUNSRN can decouple the noise and spatially varying aberration blur, avoiding the mutual influence between the denoising and non-uniform deblurring. Extensive experiments on simulation and real-world captures demonstrate that our method can significantly improve the imaging quality of single lens with large field of view (FoV), which outperforms the state-of-the-art approaches.