Deep Learning for Lung Cancer Prediction: Performance & Explainability Analysis

Globally, lung cancer continues to be among the common causes of cancer death; therefore, the importance of early patient survival increases in its detection. This study examines how deep learning can predict lung cancer using structured clinical and lifestyle survey data. A comparison was made between a multi-layer perceptron (MLP) and the random forest, XGBoost, and support vector machine (SVM) methods, as well as logistic regression machine learning models. The results demonstrate that ensemble-based models (Random Forest & XGBoost) outperformed the deep learning model, achieving an accuracy of 99.7 % compared to MLP's 94.7 %. We employed SHAP and LIME explainability techniques to improve model interpretability, identifying allergy, swallowing diffculty, and coughing as the most significant predictors of lung cancer. These results show how important it is to make AI-based medical predictions more interpretable in order to increase their use in clinical settings. While deep learning performed well, traditional machine learning models proved to be more effective for structured survey-based datasets. The study shows that machine learning could be useful for screening for lung cancer. It also stresses the need for more research on data augmentation biases, external validation on real-world clinical datasets, and advanced deep learning architectures to make the models more accurate.