Detection of pneumonia in children through chest radiographs using artificial intelligence in a low-resource setting: A pilot study

Pneumonia is a leading cause of death among children under 5 years in low-and-middle-income-countries (LMICs), causing an estimated 700,000 deaths annually. This burden is compounded by limited diagnostic imaging expertise. Artificial intelligence (AI) has potential to improve pneumonia diagnosis from chest radiographs (CXRs) through enhanced accuracy and faster diagnostic time. However, most AI models lack validation on prospective clinical data from LMICs, limiting their real-world applicability. This study aims to develop and validate an AI model for childhood pneumonia detection using Nigerian CXR data. In a multi-center cross-sectional study in Ibadan, Nigeria, CXRs were prospectively collected from University College Hospital (a tertiary hospital) and Rainbow-Scans (a private diagnostic center) radiology departments via cluster sampling (November 2023-August 2024). An AI model was developed on open-source paediatric CXR dataset from the USA, to classify the local prospective CXRs as either normal or pneumonia. Two blinded radiologists provided consensus classification as the reference standard. The model's accuracy, precision, recall, F1-score, and area-under-the-curve (AUC) were evaluated. The AI model was developed on 5,232 open-source paediatric CXRs, divided into training (1,349 normal, 3,883 pneumonia) and internal test (234 normal, 390 pneumonia) sets, and externally tested on 190 radiologist-labeled Nigerian CXRs (93 normal, 97 pneumonia). The model achieved 86% accuracy, 0.83 precision, 0.98 recall, 0.79 F1-score, and 0.93 AUC on the internal test, and 58% accuracy, 0.62 precision, 0.48 recall, 0.68 F1-score, and 0.65 AUC on the external test. This study illustrates AI's potential for childhood pneumonia diagnosis but reveals challenges when applied across diverse healthcare environments, as revealed by discrepancies between internal and external evaluations. This performance gap likely stems from differences in imaging protocols/equipment between LMICs and high-income settings. Hence, public health priority should be developing robust, locally relevant datasets in Africa to facilitate sustainable and independent AI development within African healthcare.