Research on Dimensionality Reduction Algorithms for High-Dimensional Data Based on Matrix Analysis

More and more high dimensional data can be found in many fields including genomics, computer vision, NLP, RS etc. Therefore Dimensionality Reduction is important in most of the Machine Learning and Data analysis pipelines. This article carries out a comprehensive research on different types of dimensionality reduction algorithms which are based both in linear and nonlinear paradigms. This study includes both linear and nonlinear paradigms: In this section we have discussed about the mathematical foundations of PCA, LDA, NMF, SparsePCA with respect to consider it as singular value decomposition and eigen-decompostion for the structered matrix. Regarding non-linear methods, we consider the graph - Laplacian and manifold - geometric matrix framework that is the foundation of Isomap, LLE, Laplacian eigenmaps, t - SNE, and UMAP. And we also compare computational complexity and scalability approximation(random svds, incremental). We benchmark all algorithms on commonly used datasets(MNIST, 20newsgroups, swissroll, cifar) regarding their reconstruction quality, speed and how well they enable us to classify things. For randomized svd the randomization gives us very good approximations on a small amount of the full decomposition computing time and UMAP seems to have the best mix of scale and shape when using nonlinear approaches. Proposed some practical advice for algorithm choices according to information size, internal dimension and task necessities