New/Enlarging T2 Lesions in a Progressive Multiple Sclerosis Trial Cohort: Computer vs Human Detection

Background: Although detection of new/enlarging multiple sclerosis (MS) lesions is a key metric in clinical practice and research, little is known about how human and automated detection methods compare. We compared findings made by a neuroradiologist to simulate routine practice to those made with a computer-aided technique as used in a clinical trial. Methods: MRIs from a 96-week, progressive MS clinical trial with 255 participants were evaluated for new/enlarging T2 lesions by both a neuroradiologist and a semiautomated method. Readings from 887 paired scans were compared using a paired t test and inter-rater reliability κ. Selected discordant reads were subsequently reviewed by a second neuroradiologist. Results: The semiautomated method identified new/enlarging lesions on 19.7% of the paired scans, while the neuroradiologist identified lesions on 5.7%. Of the 185 paired scans with new/enlarging T2 lesions by either method, the semiautomated method detected a mean of 3.4 new lesions, while the neuroradiologist detected 0.4. Overall κ was 0.18 (poor agreement); the κ of only scans with new/enlarging lesions by either method was –0.06. When scans were categorized as active or inactive, κ was 0.18. Unblinded neuroradiologist overread of selected discordant scans found more lesions when using advanced radiology tools such as coregistration. Conclusions: A large discordance was found between new/enlarging MS lesions identified by a neuroradiologist and a semiautomated identification method. These findings may explain clinical trials reporting new lesions in patients on highly effective therapies, while they are less common in clinical practice. These findings have implications for treatment decision algorithms using MRI. Advanced radiology tools such as coregistration may improve lesion detection in routine clinical practice.