Design of Multivariate Lung Cancer Dataset and Multistage Pre-processing to Augment Prediction of Complex Lung Cancer Datasets Using Data Mining Techniques

Progressions in genomic research have led to an increased focus on Single Nucleotide Polymorphisms (SNPs) as potential markers for various diseases, including lung cancer. This study introduces a novel approach to enhance the predictive accuracy of ensemble machine learning classifiers and design of a Multivariate Dataset for Lung Cancer for SNP-associated lung cancer through a three-stage pre-processing framework called Lung Cancer Data Pre-processing and Feature Engineering (LC-PreProFE). The framework comprises numerical analysis at the initial stage, followed by regression analysis and segmentation at the final stage objected to eliminate irrelevant features and optimize the construction of a multivariate dataset. The first stage involves rigorous numerical analysis to identify and quantify the significance of each SNP within the dataset. The stage eliminated 2 features with a 4% improvement in best predictions. The refined dataset undergoes regression analysis to model the relationships between identified SNPs and to filter out redundant or correlated features. This stage eliminated 4 features. Finally, in the segmentation process, 7 irrelevant features were eliminated. After completion of three stages, it was found that the accuracy has improved after irrelevant feature removal and the Region of Curve value reduced to show augmentation in the overall preprocessing stage.