Experience with artificial intelligence algorithms for the diagnosis of vertebral compression fractures based on computed tomography: from testing to practical evaluation

BACKGROUND: Osteoporosis is often diagnosed at the stage with complications, i.e., low-energy fractures. Vertebral compression fractures, which are complications of osteoporosis and predictors of subsequent fractures, are often asymptomatic. Compression fractures can be found by computed tomography performed for other indications with vertebral morphometry. Approaches to using artificial intelligence algorithms designed for diagnosing vertebral compression fractures were analyzed. AIM: Testing artificial intelligence algorithms to conduct morphometric analysis of vertebrae on chest computed tomography scans and assess the possibility of their implementation in medical organizations of the Moscow Healthcare Department. MATERIALS AND METHODS: To set a clinical task for artificial intelligence algorithms, basic diagnostic requirements in the area of “vertebral compression fractures (osteoporosis)” were formulated. The testing of the artificial intelligence algorithms included the following stages: self-testing, functional and calibration testing, practical evaluation, and operation testing. The first three stages of testing were performed using previously generated datasets. At practical evaluation and operation testing, artificial intelligence algorithms analyzed the data from computed tomography performed in medical organizations. The expert group of radiologists assessed the diagnostic accuracy and functional capacity of the AI algorithms at all stages. The resulting quantitative metrics of the accuracy of artificial intelligence algorithms were compared with the required target values. RESULTS: From June 2021 to June 2022, two artificial intelligence algorithms (Nos. 1 and 2) with different methods of detecting compression fractures were tested. Both artificial intelligence algorithms successfully passed the self-testing (6 tests), functional (5 tests), and calibration (100 tests) stages. The area under the ROC curve for artificial intelligence algorithm No. 1 was 0.99 (95% CI, 0.98–1), and for artificial intelligence algorithm No. 2, it was 0.91 (95% CI, 0.85–0.96). Artificial intelligence algorithm No. 1 passed the practical evaluation stage without any significant remarks, whereas algorithm No. 2 was sent for fine-tuning. After the operation testing stage, the following accuracy metrics were obtained: the areas under the ROC curve for artificial intelligence algorithm Nos. 1 and 2 were 0.93 (95% CI, 0.89–0.96) and 0.92 (95% CI, 0.90–0.94), respectively. At all stages, both artificial intelligence algorithms demonstrated sufficient metrics for clinical validation. CONCLUSION: Artificial intelligence algorithms for the automatic diagnosis of vertebral compression fractures have been tested, demonstrating the high quality of their operation. artificial intelligence algorithms can be applied as a supplementary tool in the medical decision support system.