Comparison between Principal Component Analysis and Sparse Principal Component Analysis as Dimensional Reduction Techniques for Random Forest based High Dimensional Data Classification

Random Forest has garnered a lot of attention from various researchers among numerous classification approaches. However, Random Forest performs poorly with very high dimensional data. Principal Component Analysis (PCA) and Sparse Principal Component Analysis have been suggested to enhance Random Forest’s classification performance as a dimensional reduction procedure before Random Forest is implemented. Therefore, the objective of this study was to classify high dimensional data using Random Forest alone and combined Random Forest with PCA and Sparse PCA. Using a high dimensional dataset consisting of genomic data which covers gene expression under breast cancer case (BRCA) and lung cancer case (LUAD), both PCA and Sparse PCA were applied before Random Forest was used as classification method. The results show that Sparse PCA combined with Random Forest (accuracy $=99.78 \%$) performed the best as compared to Random Forest alone (accuracy $=98.39 \%$), followed by Random Forest with PCA (accuracy $=$ 92.48%). It is noted that Sparse PCA is at advantage in tackling dimensionality problems that impede classification performance in Random Forest, particularly in genomic investigations.