OC07.02: Generalisation and deployment of an artificial intelligence system for gestational age estimation from blind sweep ultrasound to new clinical settings

Artificial intelligence (AI) systems to interpret blind sweep ultrasound (US) scans obtained with low-cost probes hold promise in expanding access to perinatal US, but their ability to generalise to new settings and probe manufacturers is unknown. Here we report our experience with adapting such an AI system to a new clinical environment. A previously trained AI model to estimate gestational age (GA) on blind sweeps in 1,978 patients from 14-36 weeks GA was embedded in a mobile app and deployed at a large, urban academic institution using a different probe manufacturer. Blind sweeps were performed by two operators without previous US experience. The model was prospectively evaluated on 1,229 sets of blind sweeps from 824 patients acquired between 16-39 weeks GA. The model's performance was compared against its performance in the original setting, both with and without model fine-tuning, and we defined a statistically significant difference in performance to be non-overlapping 95% confidence intervals. Without fine-tuning, the model's mean absolute error (MAE) for patients 16-36 weeks GA (matching original dataset) was 5.45 days (95% CI 4.51 - 6.72), which was comparable to reports of accuracy for standard clinical estimates of GA, but larger than the 3.95d (95% CI 3.58 - 4.31) reported in the original setting. After fine-tuning on only 180 exams (120 patients) from the new setting, the model's MAE for patients 16-36 weeks GA was 4.16d (95% CI 3.81 - 4.52), which was not significantly different from performance in the original setting. Our model generalised well to a new setting and probe manufacturer, achieving performance on par with standard clinical estimates, and by fine-tuning on a small number of scans (~6% of the size of the original training set), achieved similar performance to the original setting. These results validate the ability of AI-based US assessment to be deployed to new sites and expand access to care in low-resource settings.