Investigating Principal Component Analysis Impact on the Performance of Machine Learning Classifiers: A Health Informatics Application

Principal Components Analysis (PCA) is a popular dimensionality reduction and feature selection technique. It works by swiping non-informative features while retaining those that are still potentially useful. However, regardless of its wide applications in statistical analysis, the influence of PCA on the performance of classification algorithms has been rarely studied, particularly in the domain of medical image classification. In this paper, we present a thorough analysis of the influence of applying PCA on the classification of medical images. The dataset used in this study consists of thousands of images that represent four types of blood cells. These images are used to train three machine-learning classifier models after transforming images into numerical vectors using a CNN pre-trained embedding algorithm, where each image is encoded using one thousand numerical vectors. The study involves building three classification models using (1) K-nearest Neighbor (KNN), (2) Logistic Regression (LR), and (3) Random Forests (RF). Each model was built twice, once after the application of PCA to reduce the dimensionality of the embedded features and once with the original embedded features. The performance of each model is evaluated using Classification accuracy, F1, AUC, Recall, precision, and specificity metrics. The recorded performance is then compared to drawing a decisive conclusion.