Multimodal large language model assistance in interpreting postoperative calcaneal radiographs: a multicenter multi-reader study

Multimodal large language models (LLMs) may support radiographic reporting, but evidence in complex postoperative musculoskeletal imaging is limited. We compared case-level risk classification between a fixed-order LLM-assisted structured reporting session and an earlier unaided session. This retrospective multicenter multi-reader study included 73 postoperative calcaneal fixation cases from three campuses in one hospital system. Six radiologists interpreted each case unaided and, after a washout of more than 4 weeks with rerandomized order, reviewed and finalized a fixed pre-generated GPT-5 checklist and draft report in the same interface. Session order was not counterbalanced. Postoperative CT expert consensus was the pragmatic reference standard. The primary endpoint was accuracy at a binary action threshold pooling medium- and high-risk cases as requiring escalation. Across 438 case-reader observations per session, binary accuracy increased from 313/438 (71.5%) in Session 1 to 351/438 (80.1%) in Session 2 (difference, 8.7% points; cluster-bootstrap 95% CI, 1.1–17.4; OR, 1.97; 95% CI, 1.37–2.84; P < 0.001). Sensitivity increased from 54.2% to 79.2%, while specificity decreased from 85.0% to 80.9%. At the exact-high threshold, sensitivity was 45.0% and 43.3%, respectively. Mean recorded duration was 378.6 versus 98.4 s (adjusted ratio, 0.257; 95% CI, 0.248–0.266), and mean Brier score was 0.246 versus 0.177 (difference, 0.069; 95% CI, 0.007–0.147). Session 2 was associated with higher binary accuracy and shorter recorded duration, with higher sensitivity but lower specificity; sensitivity did not improve at the exact-high threshold. Because the model-populated session was always second, the study cannot separate model-generated content and report-prefill effects from order or familiarity. Prospective randomized or counterbalanced validation across independent institutions is needed.