Early stopping strategies in Deep Image Prior

Abstract The so-called Deep Image Prior approach is an unsupervised deep learning methodology which has gained great interest in recent years due to its effectiveness in tackling imaging problems. However, a well known drawback of Deep Image Prior is the need for a proper early stopping technique to prevent undesired corruptions in the reconstructed images. Determining the optimal number of iterations depends on both the specific application and the features of the images being processed. As a consequence, several numerical trials are typically required to decide when to stop the Deep Image Prior procedure, resulting in significant computational costs and time requirements. This paper aims to introduce two early stopping techniques for Deep Image Prior, based on different approaches and different perspectives. The first approach relies on the neural architecture search (NAS) strategy. The aim is to equip the neural network used in Deep Image Prior with hyperparameter configurations that produce high-quality reconstructed images that are (i) comparable to those obtained with the optimally stopped, originally configured Deep Image Prior, and (ii) achieved with significantly fewer iterations. The second proposed early stopping strategy is based on a modified version of the BRISQUE metric, a no-reference image quality measure, and it aims to track the behaviour of the PSNR curve, obtained by applying Deep Image Prior, without knowing the ground truth image. While the NAS-based early stopping technique is particularly suited in those situations where the computational time is limited, this latter one is also relevant when a larger number of iterations is allowed. Several numerical experiments on different denoising applications show a promising performance of Deep Image Prior combined with the suggested early stopping procedures.