Mitigating inter-scanner heterogeneity in brain MRI data: Assessing its impact on association analyses and the effectiveness of ComBat harmonization in multi-site neuroimaging studies

Recruiting participants from multiple sites accelerates data acquisition and increases the total sample size in neuroimaging studies, thereby enhancing the validity and generalizability of statistical findings. While both meta-analysis and mega-analysis can accommodate multi-site data, the latter leverages more effectively the high statistical power offered by large sample sizes of multi-site datasets. However, multi-site datasets often present abiotic variances stemming from differences in device manufacturer, reconstruction algorithm, acquisition parameters, and other factors, collectively termed scanner effect or inter-scanner heterogeneity. This heterogeneity hinders the application of mega-analysis and, if inadequately addressed, may obscure true effects or produce false positive effects. Furthermore, such scanner effects may vary in their impact on different brain imaging metrics (BIMs). To comprehensively understand the scanner effects on diverse BIMs, we used a multi-modal brain magnetic resonance imaging (MRI) dataset comprising 995 BIMs acquired on 28 MR scanners from two traveling subjects to characterize the inter-scanner heterogeneity (quantified by intraclass correlation coefficients) for each BIM. We then assessed the efficacy of the ComBat (combatting batch effects) harmonization method in removing such inter-scanner heterogeneity. Subsequently, using a large-scale neuroimaging dataset of 7035 subjects from CHIMGEN (Chinese Imaging Genetics), we conducted association analyses between BIMs and demographic/behavioral variables (DBVs) using both meta-analysis and mega-analysis strategies, both before and after applying ComBat harmonization, with the aim to investigate the impact of inter-scanner heterogeneity on association analyses and to evaluate the effectiveness of ComBat in mitigating this impact. The results showed that all BIMs exhibited inter-scanner heterogeneity but with varying degrees - functional connectivity (FC)-related BIMs showed the highest and cortical volume and surface area showed the lowest heterogeneity. ComBat harmonization effectively corrected the heterogeneity for most BIMs, though it was less successful for certain BIMs, particularly those related to FC. The BIM-DBV association analyses indicated that mega-analysis outperformed meta-analysis in general. However, when using uncorrected data, mega-analysis yielded an excessive number of significant, but unreliable, associations, particularly when there were only sparse associations between a DBV and a BIM in the brain. Notably, ComBat harmonization effectively addressed this issue. These results provided the first comprehensive characterization of scanner effects on extensive BIMs and new insights into the effectiveness of the ComBat harmonization technique in mitigating inter-scanner heterogeneity in multi-site neuroimaging studies to ensure the validity and reliability of statistical findings.