Hierarchical relations among principal component and factor analysis procedures elucidated from a comprehensive model

In this review article, the term “hierarchy” is related to constrained-ness, but not to superiority. Procedures A and B forming a hierarchy means that A is a constrained variant of B or vice versa. A goal of this article is to present a hierarchy of principal component analysis (PCA) and factor analysis (FA) procedures, which follows from a comprehensive FA (CompFA) model. This model can be regarded as a hybrid of PCA and prevalent FA models. First, we show how a non-random version of the CompFA model leads to the following hierarchy: PCA is a constrained variant of completely decomposed FA, which itself is a constrained variant of matrix decomposition FA. Then, we prove that a random version of the CompFA model leads to minimum rank FA (MRFA) and constraining MRFA leads to random PCA (RPCA), so as to present the following hierarchy: Probabilistic PCA is a constrained variant of prevalent FA, and the latter is a constrained variant of RPCA, which is itself a constrained variant of MRFA. Finally, this hierarchy and the above hierarchy following from the non-random version are unified into one. We further utilize the unified hierarchy to present a strategy for selecting a procedure suitable to a data set.