Validation of Artificial Intelligence in Chest X-ray Interpretation

Abstract Background Chest radiography is widely used for its portability and cost-effectiveness, providing crucial information for detecting thoracic diseases. However, interpretation challenges and the COVID-19 pandemic have led to stressed medical staff and increased misinterpretations. Automated systems, employing deep learning, are showing promise in classifying chest X-ray abnormalities. These advancements can enhance clinical workflow, decision support, and contribute to global health initiatives. Aim of the Work The study aims to evaluate the diagnostic performance of an Artificial Intelligence (AI) system in interpreting chest X-rays, specifically assessing sensitivity, specificity, and accuracy. Additionally, the research explores the impact of AI assistance on the diagnostic performance of radiology residents, examining changes in sensitivity, specificity, and accuracy. Patients and Methods A cross-sectional study at Ain Shams University Hospitals assessed the diagnostic performance of an AI system in interpreting chest X-rays, using a sample of 63 images. The study analyzed sensitivity, specificity, and accuracy, incorporating AI assistance for radiologist residents. Data management included descriptive and analytical statistics, employing ROC curves and Kappa statistics for agreement assessment. The study found diagnostic improvements with AI aid. Results The study investigated the diagnostic performance of an AI algorithm, Lunit INSIGHT CXR3, and a radiology resident in interpreting AP frontal chest X-rays. The AI demonstrated high sensitivity (100% in calcification and consolidation) and varying specificity (72-100%) and accuracy (71-100%) across different findings. The radiology resident showed significant sensitivity and accuracy improvement as well as increase in agreement with AI assistance, while no significant changes could be detected in specificity. The study suggests that AI can enhance diagnostic performance, aiding radiologists in clinical practice. Conclusion In conclusion, our study demonstrates that using AI as an aid has proven to improve the overall performance of radiology residents allowing them to reach expert like levels of diagnostic performance in the CXR interpretation.