Retrospective External Validation of Artificial Intelligence Biopsy (AIBx) Version 2 Algorithm for Thyroid Nodule Risk Stratification

Background: The Artificial Intelligence Model (AIBx) version 1 algorithm is a web-based model for analyzing thyroid ultrasound images and detecting the probability of thyroid cancer. It has demonstrated the ability to risk-stratify thyroid nodules as benign or malignant with significant accuracy. The AIBx version 2 (v2) algorithm was created by adding more images and modifying the training techniques to advance the ability to differentiate between benign and malignant thyroid nodules along with TI-RADS descriptors. This study evaluates the new algorithm's performance on an external dataset with confirmed histopathology diagnosis. Methods: All patients aged 18 years or above who underwent surgery from 2015 to 2019 were included in this study. Thyroid nodules 1–4 cm in size and with clear margins were evaluated. Thyroid lymphomas, thyroid nodules treated with ethanol ablation, and thyroid metastasis from other cancers were excluded. A longitudinal and transverse image of a thyroid nodule was provided to the AIBx v2 algorithm, with these images matched with the database of benign and malignant thyroid nodules by the AIBx v2 algorithm to classify the given nodule as either benign or malignant. The results of the AIBx v2 algorithm were compared with confirmed histopathology diagnosis to evaluate sensitivity, specificity, and positive and negative predictive values. Results: A total of 176 nodules met the inclusion criteria. The prevalence of malignancy in the dataset was 14.2%. The negative predictive value for AIBx v2 was 98.5% (95% CI, 96–100). The sensitivity, specificity, positive predictive value, and accuracy for the model were 92% (81–100), 86% (81–92), 53.5% (39–68), and 87.5% (83–92), respectively. The area under the curve was 0.894 (84–94). The diagnostic odds ratio for AIBx was 73.32. Only 4 nodules had a diagnosis of Bethesda III and IV. In this category, AIBx had a negative predictive value of 100%. Conclusion: The negative predictive value of AIBx v2 is comparable to the Bethesda System for Reporting Thyroid Cytopathology. In this study, where data from an external institution was tested, AIBx v2 showed good sensitivity and specificity. As there were only 4 nodules in the Bethesda category III and IV, further testing is needed in this area. Overall, AIBx v2 provides rapid risk stratification, which could decrease unnecessary biopsies.