Assessing inter-rater reliability of MRI features in glioma: a multi-radiologist agreement study

To evaluate the inter-rater reliability of the Visually AcceSAble Rembrandt Images (VASARI) feature set in glioma imaging assessment. Three board-certified radiologists independently analyzed 33 adult type glioma cases (20 glioblastomas, seven astrocytomas, five oligodendrogliomas, and one undefined subtype glioma) using 26 VASARI features and three additional non-VASARI features. Inter-rater reliability was assessed using Fleiss’ kappa and Gwet’s AC with conditional confidence intervals. Ordinal weighting was applied to relevant features. High inter-rater reliability was observed for multifocality, enhancement quality, cyst presence, enhanced tumor crossing of midline, and calvarial remodeling. There was poor agreement for diffusion characteristics, hemorrhage presence, involvement of eloquent brain regions, and T1/FLAIR ratio. Several features, including ependymal invasion and tumor size, showed borderline reliability. Inter-rater reliability varied across the 29 features evaluated for glioma imaging. Eleven features demonstrated high reliability, and four showed poor agreement. The remaining features exhibited varying levels of reliability, with some falling in a borderline range, indicating areas for potential improvement in standardization and assessment. N/A.