Evaluating Multimodal Vision Frontier Models for Grading Neuroforaminal Stenosis on Lumbar Spine MRIs

Study Design Diagnostic accuracy study. Objective To determine whether a general-purpose multimodal artificial intelligence (AI) model, Gemini 3 Pro, grades lumbar neuroforaminal stenosis on magnetic resonance imaging (MRI) in agreement with expert radiologists, particularly for severe stenosis, and whether a vertebral-level localizer overlay helps. Methods One hundred lumbar MRIs from the Radiological Society of North America (RSNA) 2024 Lumbar Spine Degenerative Classification Dataset, graded by expert radiologists, provided 1,000 foramina (five levels, both sides) labeled normal/mild, moderate, or severe. Sagittal and axial T2-weighted series were converted into short scrolling videos and submitted to the model with a structured grading prompt; no task-specific training was performed. The primary outcome was agreement with expert grades (linearly weighted Cohen’s kappa, κw); secondary outcomes were accuracy, severe-stenosis sensitivity, direction of errors, and the localizer effect. Results Agreement with expert grading was poor (κw 0.20, 95% confidence interval [CI] 0.14–0.26). Accuracy was 70.6% (95% CI 67.7–73.4) but reflected the 70.4% of foramina that were normal/mild. The model identified 17.9% of severe foramina (95% CI 8.9–30.4) and called 58.9% normal/mild. Errors were systematically toward undergrading (p < 0.001). The localizer did not improve accuracy (70.0% vs 70.6%, p = 0.590) and reduced severe sensitivity (3.6% vs 17.9%, p = 0.043). Conclusions Without task-specific training, a general-purpose AI model reached moderate accuracy only because most foramina were normal or mild; it missed most severe stenosis and systematically undergraded. In its current form this approach is unsuitable for screening, triage, operative decision-making, or independent report generation.