Enhancing Diagnostic Accuracy in Lung Cancer Detection Using Diverse Classification Models

Abstract— Lung cancer is a leading cause of death and early identification of lung cancer is crucial for improved treatment and survival. The traditional diagnostic methods are mostly imaging and clinical assessment that can sometimes have drawbacks including late diagnosis, subjectivity, and a lack of predictive value. A comprehensive lung cancer risk dataset is used and preprocessed in this study, using normalisation methods (StandardScaler and MinMaxScaler) and feature selection (Recursive Feature Elimination with Support Vector Machines). The SMOTE was used to overcome class imbalance. RandomizedSearchCV with KFold cross validation was used to train and optimize various ML regression and classification models such as LR, KNN, Gaussian NB, Multinomial NB, DT, SVC, RF, XGBoost, GB and MLP. Implemented a Voting Classifier which is a combination of Random Forest, Decision Tree and Extra Trees. Improved the interpretability of the model using used explainable AI methods like LIME and SHAP. Deployed using Flask framework. The evaluation using accuracy, precision, recall, f1 score and confusion matrix indicated that VC achieved 98.1% on all the criteria compared to all the classifiers considered individually. Combining ensemble learning and explainable AI enhances diagnostic reliability, transparency, and interpretability, greatly boosting the accuracy of lung cancer detection compared to traditional methods.