Singular Value Versus Eigenvalue Decomposition Efficiency in Computing Principal Components for Dimensionality Reduction of Large Datasets

Principal component analysis, being one of the best techniques for dimensionality reduction, is implemented by using one of the two high-accuracy algorithms: the singular value decomposition (SVD) and eigenvalue decomposition (EVD). The EVD is generally faster than the SVD, except for datasets with fewer observations or when the observation has fewer features. Apart from cases of shallower datasets consisting of just a few hundred double-precision observations, the EVD speeds up computing principal components by at least 4.5%, whereas the average speedup in 45% widely varies from 12% to 92%. The speedup on non-shallower single-precision datasets is roughly similar, but it nonetheless makes no sense due to EVD poor accuracy while operating on numeric data with single precision. The EVD is efficient if the dataset consists of no fewer than a few hundred observations (objects) having at least three double-precision features.