THREE SYSTEMS, ONE DIAGNOSIS: WHAT WE LEARNED FROM TESTING PERIPROSTHETIC JOINT INFECTION CLASSIFICATIONS IN A CASE-BASED STUDY

Aim To evaluate the agreement between users and the original authors (gold standard - GS) across three periprosthetic joint infection (PJI) classification systems – McPherson, JS-BACH, and PJI-TNM – while also assessing their perceived usability, clarity, and user preference. Method Ninety-two specialists classified 10 anonymized PJI cases using the three systems. For each subcomponent, we calculated agreement with the gold standard (GS) and measured interobserver agreement using percentage agreement. A 70% agreement threshold was used to define acceptable concordance. A parallel questionnaire assessed perceived clarity, usability, preference, and time burden. Results JS-BACH demonstrated the highest mean agreement with the GS (80,1%), followed by McPherson (73,9%) and PJI-TNM (56,5%). The subcomponents with less than 70% agreement were lowest in JS-BACH (9/40) and highest in PJI-TNM (17/30). Overall, the agreement between GS and modal user response was generally high but diverged in components such as ‘Host’ and ‘Local Extremity’. Interobserver agreement reflected these trends: JS-BACH had the most consistent user responses, followed by McPherson. PJI-TNM displayed substantial variability, suggesting challenges in clarity or applicability. Subjectively, JS-BACH was rated as the most useful classification (49,4%), the easiest to understand (38,6%), and the easiest to use (3,6/5). McPherson was favored by 28,9% and rated second for clarity and usability (33,7% and 3,3/5, respectively), while PJI-TNM ranked lowest across all user domains. Mean Likert scores (1–5) for the classifications’ subcomponents were highest for JS-BACH (3,8), followed by McPherson (3,5) and PJI-TNM (3,2). As for time to completion, McPherson was the quickest overall (3,2 minutes per case), closely followed by JS-BACH (3,3 minutes) and PJI-TNM (3,7 minutes). Conclusions JS-BACH demonstrated the best alignment between expert-defined GS and the overall responses to the questionnaire, as well as the highest interobserver consistency and superior subjective usability. McPherson performed moderately across all domains. PJI-TNM revealed limitations in both reproducibility and perceived clarity. Nonetheless, our findings also suggest a need to revise ambiguous subcomponents in the systems that scored below 70% in the agreement between the GS and the modal user responses.