A Proximal Variable Smoothing for Nonsmooth Minimization Involving Weakly Convex Composite with MIMO Application

We propose a proximal variable smoothing algorithm for nonsmooth optimization problem with sum of three functions involving weakly convex composite function. The proposed algorithm is designed as a time-varying forward-backward splitting algorithm with two steps: (i) a time-varying forward step with the gradient of a smoothed surrogate function, designed with the Moreau envelope, of the sum of two functions; (ii) the backward step with the proximity operator of the remaining function. For the proposed algorithm, we present a convergence analysis in terms of a stationary point by using a newly smoothed surrogate stationarity measure. As an application of the target problem, we also present a formulation of multiple-input-multiple-output (MIMO) signal detection with phase-shift keying. Numerical experiments demonstrate the efficacy of the proposed formulation and algorithm.