How green are large language models for radiology report labelling? Comparing human, rule-based and hybrid workflows

Abstract Objectives To address limited quantitative data on sustainable use of large language models (LLMs) in radiology, we quantified the resource footprint of LLMs for labelling CT pulmonary embolism reports and assessed how a hybrid rule-based–LLM workflow changes time, cost and carbon emissions compared with manual labelling. Materials and methods In this single-centre retrospective study, 2923 structured CT reports were labelled using four workflows: a rule-based extractor (RBE), an LLM-only pipeline using 18 open-weight and four proprietary models, a hybrid RBE–LLM pipeline that routed RBE failures to an LLM, and full manual labelling by radiologists. Ground truth was based on radiologist adjudication. For each LLM, we measured per-report latency, estimated CO 2 emissions and cost. Radiologists recorded the labelling time per report. Results Manual labelling required 32.8 h for 2923 reports (40.4 s/report; €0.42/report) with 95.0% accuracy (95% CI: 93.7–96.2). LLM-only pipelines were less accurate (85.1%; 95% CI: 84.9–85.5) but reduced labelling time to 12.4 h and cost to €2.60 (both p < 0.001). Hybrid RBE–LLM workflows yielded the highest accuracy (98.5%) and lowest resource use: across 22 models, switching from LLM-only to hybrid reduced time (6.7 to 0.97 h), cost (€1.19 to €0.17), and CO 2 (0.82 to 0.12 kg; all p < 0.001). Conclusion LLM-only labelling reduced labour time and direct costs compared with manual annotation. A hybrid RBE–LLM pipeline that forwards rule-based failures to an LLM concentrated compute where needed and markedly decreased time, cost and emissions, supporting targeted deployment of LLMs for sustainable data-annotation workflows in radiology. Critical relevance By quantifying time, cost and carbon emissions of manual, rule-based, LLM and hybrid report labelling, this study identifies sustainable workflows for deploying LLMs in routine radiology reporting. Key Points Manual expert labelling of CT pulmonary embolism reports is time-intensive and costly. Mid-sized LLM configurations provide favourable trade-offs between performance and resource use. Hybrid rule-based–LLM workflows sustain accuracy while reducing resource demands.