Enhanced cancer prognosis with machine learning-driven feature selection, optimization and classification in high-dimensional data

This paper introduces an advanced machine learning framework for improved cancer diagnosis using optimized feature selection, robust classification and increased interpretability on high dimensional microarray data. In the proposed approach, the linear discriminant analysis (LDA) is used for feature selection, ant lion optimizer (ALO) is used for optimization, a random forest (RF) is used for classification, and an XAI is used for model transparency and biological understanding. Three different cancer datasets (Breast Cancer, Lung Cancer, and Ovarian Cancer) with thousands of gene expression profiles in various sample types (cancerous and non-cancerous tissues) were used. LDA is used to simplify the high-dimensional data, separating the most meaningful features and ALO is employed to improve the prediction results by selecting the optimal feature subset. RF demonstrated good classification of types of cancers, and XAI helped in understanding the classification by showing the most significant genes contributing to the prediction. This new framework showed high accuracy for all the datasets and promising results for discrimination between cancer and no-cancer tissues as well as for different relapse conditions with increased transparency and clinical confidence in the decision-making process. The research employs an intelligent combination of methods to select significant data and make the cancer detection more accurate. It has good efficiency in breast, lung and ovarian cancer specimens, giving uniform and dependable outcomes. The method also describes what genes are used to make the predictions, which can be used to develop confidence to use clinically.