A novel approach to integrating Vision Transformers and machine learning for robust lung nodule classification using CT imaging

To develop and compare a lung nodule classification framework using CT images, evaluating an end-to-end Vision Transformer, a feature extraction pipeline, and a radiomics-based method for diagnostic accuracy. A dataset of 435 patients with confirmed lung nodules (241 benign, 194 malignant) was used. CT images were normalized, resized, and segmented with manual corrections by radiologists. Three approaches were implemented: (1) an end-to-end Vision Transformer integrating CNN layers with transformer encoders and patch embedding; (2) a deep feature extraction pipeline feeding Vision Transformer features into ML classifiers (SVM, Random Forest, XGBoost, CatBoost, TabNet, TabTransformer); and (3) a radiomics-based method extracting 215 handcrafted features from segmented nodules. Feature selection (PCA, RFE, MI) and hyperparameter tuning via grid search were applied. Models were trained using 5-fold cross-validation and evaluated on internal and an independent external test set of 99 patients (54 benign, 45 malignant) to confirm generalizability. The end-to-end Vision Transformer achieved 94 % accuracy, 93 % sensitivity, and 96 % AUC on the training set, with internal test performance of 92 % accuracy, 91 % sensitivity, and 93 % AUC, and external test performance of 91 % accuracy, 90 % sensitivity, and 92 % AUC. Deep feature extraction using TabTransformer and TabNet yielded accuracies between 92 % and 95 % and AUCs up to 93 %. The radiomics-based approach, especially when combined with feature selection, produced comparable AUC values. Statistical tests confirmed significant performance differences, while SHAP analysis identified texture and intensity as key discriminative features. These results demonstrate that deep learning and radiomics approaches effectively capture the heterogeneous characteristics of lung nodules, providing reliable diagnostic performance across diverse datasets. Our findings confirm that transformer-based and hybrid analytical approaches yield high diagnostic performance for lung nodule classification. Future studies should integrate multi-modal data to further enhance clinical utility.