Effect of online training on the reliability of assessing sacroiliac joint radiographs in axial spondyloarthritis - a randomized, controlled study

OBJECTIVE Radiographic assessment of sacroiliac joints (SIJs) according to the modified New York (mNY) criteria is key in classification of axSpA but has moderate inter-reader agreement. We aimed to investigate the reliability improvements scoring SIJ radiographs after applying an online real-time iterative calibration (RETIC) module, in addition to a slideshow and video alone. METHODS Nineteen readers, randomized to 2 groups (A/B), completed 3 calibration steps: I) review of manuscripts, II) review of slideshow and video and group A completed RETIC, III) re-review of slideshow+video and group B completed RETIC . The RETIC module gave instant feedback on reader's gradings and continued until predefined reliability (kappa) targets for mNY positivity/negativity were met. Each step was followed by scoring different batches of 25 radiographs (Exercises I-III). Agreement (kappa) with an expert radiologist was assessed for mNY+/mNY- and individual lesions. Improvements by training strategies were tested by linear mixed models. RESULTS In exercises I/II/III, mNY kappas were 0.61/0.76/0.84 in group A, and 0.70/0.68/0.86 in group B, respectively, i.e. increasing, mainly after RETIC completion. Improvements were observed for both grading mNY+/mNY- and for individual pathologies, both in experienced and, particularly, inexperienced readers. Completion of the RETIC module in addition to slideshow and video caused a significant kappa increase of 0.17 (CI: 0.07-0.27, P=0.002) for mNY+/mNY- grading, while completion of slideshow and video alone did not (0.0; CI -0.10-+0.10, P=0.99). CONCLUSION Agreement on scoring radiographs according to the mNY criteria significantly improved when adding an online RETIC module, but not by slideshow and video alone.