Enhancing Automated Lung Disease Detection from Chest Radiography: An Interpretable Deep Learning Approach with Integrated Preprocessing and Ensemble Modeling

In this study, we tackled the problem of early pulmonary disease detection in chest radiography by developing and evaluating a sophisticated deep learning framework. Our goal was to improve diagnostic precision for conditions like cancer and pneumonia. To do this, we combined insights from current literature with a rigorous experimental protocol using a 5,000-image sample from the well-known NIH Chest X-ray dataset. A core component of our work was the implementation of a multi-stage preprocessing pipeline, featuring both lung segmentation and targeted contrast enhancement, designed to focus the model’s attention on relevant clinical features. We assessed our models, which were trained as an ensemble of specialized classifiers, using a suite of standard metrics (accuracy, sensitivity, specificity, and AUC). Our findings show a clear advantage for our deep convolutional neural network (CNN) approach, which achieved an 82.4% average accuracy and a 0.89 AUC. We demonstrated that our unique preprocessing steps were highly effective, boosting accuracy by a substantial 13.2%. We also critically discuss the limitations of our work, including challenges with model generalizability and the smaller dataset size compared to prior studies with higher reported accuracies. In conclusion, our work supports the move towards AI-driven precision radiology and lays out key recommendations for future research, such as creating more transparent systems and performing multi-center validation.