Automatic detection of simulated artifacts on T1w magnetic resonance images: Comparing performance of different QC strategies

The reliability of MRI-derived measures critically depends on image quality. Poor-quality scans can obscure anatomical detail and compromise the accuracy of automated image analysis, underscoring the need for robust quality control (QC) procedures. Automated QC offers scalability for large neuroimaging datasets, yet the comparative performance of different approaches for detecting specific artifact types remains poorly understood. We systematically compared rule-based (RB), classical machine learning (ML), and deep learning (DL) QC algorithms using 1,000 high-quality T1w scans. Four artifact types, blurring, ghosting, motion, and noise, were synthetically introduced across ten severity levels using TorchIO, yielding 40,000 degraded images. Visual QC of a subset confirmed strong inter-rater reliability (Krippendorff’s α = 0.82, mean Spearman’s ρ = 0.87). RB and ML models used 62 image quality metrics from MRIQC, whereas DL models were trained directly on minimally preprocessed images. Models were trained with participant-level five-fold cross-validation and tested on an independent dataset. DL models achieved the highest overall performance across simulated artifact types (Youden’s Index [YI] = 0.83–0.97). RB and ML performed comparably at high artifact severities (YI ≥ 0.75) but showed limited sensitivity to subtle simulated ghosting and noise (YI ≤ 0.15). Feature analysis indicated that RB relied primarily on normative metrics, whereas ML flexibly adapted feature use by artifact type and severity. These findings highlight DL’s superior generalizability for detecting subtle simulated artifacts and provide a proof-of-concept for selecting QC strategies for similar benchmarking tasks.