Variable Smoothing Alternating Proximal Gradient Algorithm for Coupled Composite Optimization

Abstract In this paper, we consider a broad class of nonconvex and nonsmooth optimization problems, where one objective component is a nonsmooth weakly convex function composed with a linear operator. By integrating variable smoothing techniques with first-order methods, we propose a variable smoothing alternating proximal gradient algorithm that features flexible parameter choices for step sizes and smoothing levels. Under mild assumptions, we establish that the iteration complexity to reach an $$\varepsilon $$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mi>ε</mml:mi> </mml:math> -approximate stationary point is $$\mathcal {O}(\varepsilon ^{-3})$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mi>O</mml:mi> <mml:mo>(</mml:mo> <mml:msup> <mml:mi>ε</mml:mi> <mml:mrow> <mml:mo>-</mml:mo> <mml:mn>3</mml:mn> </mml:mrow> </mml:msup> <mml:mo>)</mml:mo> </mml:mrow> </mml:math> . The proposed algorithm is evaluated on sparse signal recovery and image denoising problems. Numerical experiments demonstrate its effectiveness and superiority over existing algorithms.