Assessing the reliability of Response Evaluation Criteria In Solid Tumors (RECIST): a systematic review of the factors contributing to inter-observer variability

BACKGROUND: The Response Evaluation Criteria in Solid Tumors (RECIST) have been the standard for assessing tumor response in oncology trials since 2000. Despite the use of these criteria, interobserver variability (IOV) remains a significant concern, as it can affect patient management and clinical trial outcomes. We aimed to review the current literature on factors contributing to IOV in RECIST. MATERIALS AND METHODS: We conducted a systematic review to summarize potential factors that can contribute to variability, variance, or reproducibility in RECIST assessments: (I) manual measurements, (II) selection of lesions, (III) 1D diameters, (IV) intra-radiologist variability, (V) the experience of the readers, (VI) local vs centralized assessment, (VII) variations of criteria used, (VIII) type of imaging, (IX) follow-up schedule. Inclusion criteria required studies to use RECIST (1.0, 1.1, or modified versions), involve multiple response evaluations and provide comparative data on the same test subjects. RESULTS: We identified 246 studies, of which 88 met our inclusion criteria. Median sample size across studies was 50 patients (IQR: 25.8-90.8). Most studies (68.2%) used CT scans as the sole imaging modality. The most common primary tumors studied were lung and hepatocellular carcinoma (each 14.8%). Most studies showed an IOV being affected by measurement methods, lesion selection, and imaging quality. Statistical methods for IOV vary: Kappa for categorical and intraclass correlation for continuous data, with calculation differences limiting comparability and risking misleading conclusions. CONCLUSION: Standardized protocols are needed to study interobserver variability in RECIST and enable us to understand the current criteria and develop better ones. KEY POINTS: We should standardize the assessment of interobserver variability in RECIST to improve reproducibility, comparability, and clinical trial decision reliability. Interobserver variability in RECIST depends on measurement methods, lesion selection, and imaging quality. Inconsistent statistics hinder study comparability. Standardized RECIST protocols can reduce variability, improving evaluation consistency, clinical decisions, and treatment outcomes in oncology trials.