Assessment of inter-reader agreement and diagnostic performance of Node-RADS in colon cancer

Abstract Objectives To assess the inter-reader agreement and diagnostic performance of Node-reporting and data system (Node-RADS) in the evaluation of colon cancer among radiologists with varied experience levels. Materials and methods A total of 247 patients with colon cancer were included in this retrospective study. Lymph node evaluations were performed by six radiologists with diverse experience levels (three junior and three senior readers) using Node-RADS. Inter-reader agreement for Node-RADS classification and specific imaging features was assessed using the kappa statistic and compared between junior and senior readers. The diagnostic performance of Node-RADS was evaluated at a cutoff of 3 and 4 score. Results Lymph node metastasis was observed in 47.8% of patients. The overall inter-reader agreement was substantial for Node-RADS classification (k = 0.69), almost perfect for size (k = 0.83), moderate for texture and border (k = 0.41–0.45), and fair for shape (k = 0.37). Senior readers demonstrated higher inter-reader agreement in both Node-RADS and specific features than junior readers. Senior readers achieved higher AUC (0.79–0.80) than junior readers (0.72–0.78). At a cutoff of 3, sensitivity, specificity, and accuracy across all readers were 52.5–70.3%, 84.5–93.0%, and 69.2–79.8%, respectively. The corresponding values were 33.9–44.9%, 92.3–98.4%, and 64.4–72.5%, respectively, at a cutoff of 4. Senior readers showed significantly higher pooled sensitivity and accuracy, with comparable specificity compared to junior readers at both Node-RADS cutoffs. Conclusion The Node-RADS demonstrates high agreement between observers. Senior readers showed higher inter-reader agreement and diagnostic performance than junior readers. Node-RADS exhibits high specificity but relatively low sensitivity for detecting lymph node metastasis in colon cancer. Critical relevance statement This study demonstrated good inter-observer agreement for Node-reporting and data system (Node-RADS) in colon cancer across readers of varying experience, supporting its clinical reliability. However, its limited sensitivity for predicting lymph node metastasis highlights the need for further improvement. Key Points Assessing inter-observer agreement among different readers for Node-RADS assignment is critical before its clinical application. Substantial inter-observer agreement for Node-RADS was observed across readers of varying experience levels.