Impression-regularized finetuning enables small language models to extract cervical spine fractures from radiology reports

We assessed whether small, open-weight large language models (LLMs) can accurately extract vertebral-level acute cervical spine fracture labels from CT radiology reports while supporting on-premises deployment, and whether impression-regularized finetuning improves accuracy, data efficiency, and temporal stability. In this single-center retrospective study, 8,567 adult cervical spine CT reports (2017–2022) were labeled for acute occipital condyle and C1–C7 fractures with neuroradiologist oversight. Reports from 2017 (n = 1,064) were used for finetuning and reports from 2018–2022 (n = 7,503) for temporal testing. We finetuned Gemma-3 (1B/4B/12B), Llama-3.2-3B, and Phi-3.5-Mini using standard low-rank adaptation (LoRA) and with an impression-regularized LoRA approach. During training, the model samples multiple synthetic targeted impressions focused on acute cervical spine fractures, selects the candidate most consistent with the reference standard labels, and applies an auxiliary impression loss as regularization. At inference, all models perform single-pass deterministic JSON extraction. Baselines included zero-shot prompting, two-pass impression-guided prompting, and standard LoRA. Impression-regularized LoRA improved Gemma-3-4B macro-F1 from 0.885 to 0.909 (macro-precision 0.929, macro-recall 0.893) while reducing variance across training runs. Improvements were largest for smaller models and in low-data settings; Gemma-3-1B macro-F1 increased from 0.646 to 0.782. For this vertebral-level acute cervical spine fracture extraction task, impression-regularized Gemma-3-4B approached the zero-shot performance of the Gemma-3-27B (macro-F1 0.909 vs. 0.913). Across the 2018–2022 temporal evaluation, mean F1 increased from 0.871 to 0.893 with similar or lower variability across time windows. Systematic error analysis showed that remaining errors were mainly associated with ambiguous report language and label-boundary cases. Overall, impression-regularized LoRA enables small open-weight LLMs to accurately extract acute cervical spine fracture labels while maintaining performance across a multi-year test period and preserving auditable single-pass inference suitable for privacy-preserving on-premises deployment.