Multimodal machine learning for predicting spine surgery recommendation using radiology reports, clinical notes, and structured clinical data

Objective With the need for efficient triaging methods in spine clinic increasing, machine learning and natural language processing (NLP) transformer models have demonstrated effectiveness in predicting surgical recommendations, offering a potential approach for patient stratification. We evaluated an integrated fusion model, trained on radiology reports, clinical notes, and clinical variables, to predict spine surgery recommendations in an all-comer population. Methods A single-institution cohort of spine clinic patients with over 50 ICD-coded spine pathologies was analyzed. From initial clinic visits, we collected radiology reports, clinic notes, structured clinical variables, and surgery recommendation status within one year. Separate unimodal models were developed for each data modality: radiology impressions, clinic notes, and structured clinical variables. Outputs from component models were subsequently integrated using logistic regression to create late-fusion models. We developed seven late-fusion models incorporating different NLP transformer backbones (e.g., BERT and GatorTRON). Performance was assessed using 5-fold cross-validation with area under the receiver operating characteristic curve (AUROC), sensitivity, and specificity with 95% confidence intervals. Results Within one year of initial clinic visit, 35% of 972 patients were offered spine surgery. Late-fusion models exhibited varying performances in predicting spine surgical recommendation (AUROC range: 0.69 – 0.81). The GatorTRON late-fusion model was the best performer (AUROC 0.81; 95% CI 0.78 – 0.83). Furthermore, among the three component modalities, the radiology-only model provided the largest contribution to late-fusion model performance. Conclusions Our investigation revealed that an integrated machine learning model can robustly predict spine surgical recommendation from multimodal data available at initial spine clinic visits.