Agreement and workflow efficiency of AI-based coronary artery calcification quantification in lung cancer screening: Comparison with semi-automated and visual assessment

Objectives To evaluate agreement and workflow implications of fully automated AI-based coronary artery calcification (CAC) quantification on non-ECG-gated low-dose CT in lung cancer screening, compared with semi-automated (SA) and visual assessment. Materials and methods In this retrospective single-center study, 323 participants (55.7% male; median age 61 years; 52–79 years) undergoing low-dose CT for lung cancer screening were included. CAC was quantified using SA and AI-based Agatston scoring. Two readers performed visual grading. Agreement between SA and AI was assessed using intraclass correlation coefficient (ICC), Spearman correlation, and Bland-Altman analysis. Categorical agreement (CAD-RADS 2.0 plaque burden) and CAC detection were evaluated using weighted Cohen's κ and diagnostic metrics. CAC processing times for were compared. Results AI-based and semi-automated Agatston scores showed excellent agreement (ICC 0.96) and strong correlation (Spearman r = 0.97), with a small bias (21.5) and moderate limits of agreement. AI achieved high diagnostic performance for excluding CAC (sensitivity 0.97, 95%-CI, 0.92–0.99; specificity 0.91, 95%-CI, 0.86–0.94). Categorical agreement between AI and SA was almost perfect (κ 0.92) and higher than agreement between SA and visual assessment (κ 0.84 and 0.68). SA scoring required substantially longer processing time (102.0 ± 95.7 s) compared with visual assessment (15.5 ± 6.0 s and 24.2 ± 7.4 s; p < 0.001). Conclusion AI-based CAC quantification on non-ECG-gated low-dose CT demonstrates excellent agreement compared to semi-automated scoring, with higher categorical agreement than visual assessment and no requirement for manual scoring. AI-based approaches may facilitate standardized and scalable CAC reporting in lung cancer screening without additional reading time.