A novel nonconvex penalty method for a rank constrained matrix optimization problem and its applications

The rank constrained nonconvex nonsmooth matrix optimization problem is an important and challenging issue. To solve it, we first design a penalty model in which the penalty term can be expressed as a sum of specific functions defined on smallest singular values of the matrix in question. We prove that the global minimizers of this penalty model are the same as those of the original problem. Second, we propose a flexible factorization format for the penalty function, such that the model enjoys the merit of fast computation in a SVD-free manner. We further prove that the factorization format problem is equivalent to the penalty one. A Bregman proximal gradient (BPG) method is developed for optimizing the factorization model. Third, we use two application problems as examples to illustrate that the problem considered has a wide application. Finally, some numerical experiments are conducted, and their results indicates the effectiveness of the proposed method.