Artificial Intelligence–Assisted ACR TI-RADS and Bethesda-like Risk Prediction for Indeterminate Thyroid Nodules: A Retrospective Proof-of-Concept Evaluation

Background/Objectives: Cytologically indeterminate thyroid nodules (Bethesda III–IV) remain a diagnostic challenge. We evaluated the TRAICK (version 1.9.4; TRAICK, Istanbul, Türkiye) artificial intelligence (AI) system, which generates an American College of Radiology Thyroid Imaging Reporting and Data System (ACR TI-RADS) category and an imaging-derived Bethesda-like risk category from gray-scale ultrasound images. Methods: This retrospective proof-of-concept study included 52 surgically resected nodules with Bethesda III (n = 29) or IV (n = 23) cytology, with histopathology of the index nodule as the reference standard. The two TRAICK outputs were analyzed separately, and the ACR TI-RADS output was compared with two radiologists using the McNemar and DeLong tests. Results: Twenty-three nodules (44.2%) were malignant on index-nodule histopathology. The TRAICK ACR TI-RADS output had an accuracy of 71.2% (95% CI 57.7–81.7) and an area under the ROC curve (AUC) of 0.762 (0.640–0.884). Its AUC did not differ significantly from that of Radiologist 1 (0.762 vs. 0.777, p = 0.783); specificity was higher (65.5% vs. 37.9%, p = 0.021), whereas sensitivity was lower without a statistically significant difference (78.3% vs. 91.3%, p = 0.250). Compared with Radiologist 2, the TRAICK ACR TI-RADS output had higher accuracy (71.2% vs. 51.9%, p = 0.013), specificity (65.5% vs. 24.1%, p < 0.001), and AUC (0.762 vs. 0.629, p = 0.042). The Bethesda-like output had an AUC of 0.822 (0.707–0.938), with a sensitivity of 60.9% and specificity of 86.2% at the V–VI threshold. In sensitivity analyses, the AUC of the ACR TI-RADS output did not differ significantly from that of Radiologist 1, whereas its AUC advantage over Radiologist 2 was not statistically significant when low-risk neoplasms were classified as malignant. Conclusions: In this small, selected surgical cohort, the TRAICK ACR TI-RADS output showed discrimination that did not differ significantly from that of the more accurate of two experienced radiologists, with higher specificity, and its Bethesda-like output showed promising discrimination. These findings support further study of TRAICK as a complementary aid for preoperative decision-making; independent prospective validation is needed.