Optimality conditions for penalized sparse PCA

Abstract This paper establishes the theoretical foundations of an alternating optimization scheme for penalized sparse principal component analysis (PCA) focusing on variance maximization. We provide a theoretical foundation for the optimality of solutions derived from this widely used algorithm, addressing a gap in the current literature where empirical results often lack theoretical support. We show the algorithm’s success when the dataset’s covariance matrix is positive definite. Additionally, we characterize sparsity-inducing penalties and examine the use of various ones, including the $$ \ell _1$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:msub> <mml:mi>ℓ</mml:mi> <mml:mn>1</mml:mn> </mml:msub> </mml:math> -norm, SCAD, and $$\ell _0$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:msub> <mml:mi>ℓ</mml:mi> <mml:mn>0</mml:mn> </mml:msub> </mml:math> -norm. We conduct numerical experiments to evaluate standard metrics, such as explained variance, number of iterations, and computational time.