Enhancing Predictive Analytics in Big Data through Feature Selection and Dimensionality Reduction Techniques

The rising sophistication of Big Data has been marked by high dimensionality that poses numerous challenges when predicting analytics computations and model performance. This paper tries to adopt an integrated features selection and dimensionality reduction based on Recursive Feature Elimination (RFE) and Principal Component Analysis (PCA) respectively. The removal of those features and the use of principal components analysis lead to a more accurate predictive model while reducing the need for computation time. Furthermore, experimental results indicate that the level of accurate prediction in the model is enhanced from 75% to 85% in addition to the decrease of computation time by 20%. The approach successfully overcomes the obstacles of overfitting and increased computational costs associated with processing big data. This method is particularly applicable for call centers, hospitals, banks, credit cards, insurance, or any other domains, where large data volumes are dealt with frequently and where large volumes of data can be a key differentiator.