Comparative Analysis of Machine Learning and Deep Learning Models for Lung Cancer Prediction Based on Symptomatic and Lifestyle Features

Lung cancer remains a leading cause of global mortality, with early detection being critical for improving the patient survival rates. However, applying machine learning and deep learning effectively for lung cancer prediction using symptomatic and lifestyle data requires the careful consideration of feature selection and model optimization, which is not consistently addressed in the existing research. This research addresses this gap by systematically evaluating and comparing the predictive efficacy of several machine learning and deep learning models, employing rigorous data preprocessing, including feature selection with Pearson’s correlation, outlier removal, and normalization, on a patient symptom and lifestyle factor dataset from Kaggle. Machine learning classifiers, including Decision Trees, K-Nearest Neighbors, Random Forest, Naïve Bayes, AdaBoost, Logistic Regression, and Support Vector Machines, were implemented using Weka simultaneously with neural network models with 1, 2, and 3 hidden layers, which were developed in Python within a Jupyter Notebook environment. The model performance was assessed using K-fold cross-validation and 80/20 train/test splitting. The results highlight the importance of feature selection for enhancing the model accuracy and demonstrate that the single-hidden-layer neural network, trained for 800 epochs, achieved a prediction accuracy of 92.86%, outperforming the machine learning models. This study contributes to developing more effective computational methods for early lung cancer detection, ultimately supporting improved patient outcomes.