SN-NET: Semismooth Newton Driven Lightweight Network for Real-World Image Denoising

Semismooth Newton is a powerful tool to tackle the regularization problems in image restoration. Compared to other optimization methods such as alternating direction method of multipliers (ADMM), the semismooth Newton method exhibits faster convergence and greater efficiency, since the non-linear and coupling system is solved simultaneously. However, its performance relies heavily on the handcrafted parameters and efficient solvers for calculating Newton steps. To tackle this issue, we first develop an improved semismooth Newton method, in which we turn the original nonlinear system solving problem into a network-friendly convex optimization problem. After that, we unfold it into a novel network namely SN-Net. We apply SN-Net on the most fundamental image denoising task, which shows great advantages in the following two aspects. (1) The network is quite lightweight, i.e., the number of network parameters is only 86 KB. (2) The network structure exhibits strong interpretability. To the best of our knowledge, the $\mathrm{SN}-\mathrm{Net}$ is the first attempt to successfully map the semismooth Newton method to a learnable network. The great success of it on image denoising may inspire many potential works on other image restoration tasks. The code and the pre-trained models are released at https://github.com/pandazcx/SN-Net.