An inter-rater reliability study of a modified version of SATS as a prehospital triage tool

BACKGROUND: Triage systems play an essential role in allocating patients to an appropriate level of care. Several triage systems are used worldwide, all of which require scientific evaluation of their reliability and validity. Evidence regarding prehospital triage systems remains limited. The South African Triage Scale (SATS) has been adapted for local use in several regions, but modified versions require independent evaluation. The objective of this study was to examine the reliability of a modified version of the triage system, SATS, when it is used in the emergency medical services (EMS) in a region in Sweden. The study also investigated whether triage performance differs among registered nurses with varying experience and competence levels. METHODS: This prospective cross-sectional inter-rater reliability study included 34 registered nurses (RNs) working in the regional EMS. The participants independently triaged 30 prehospital case vignettes via a digital questionnaire, resulting in 1,010 triage assessments. Descriptive data were used. Inter-rater reliability was assessed via Krippendorff's alpha (α) and percentage agreement. Mixed effects logistic regression was performed to explore associations between triage assignment and RN characteristics, including gender, years of experience and specialist education. RESULTS: This study indicated insufficient inter-rater agreement when a modified version of SATS was used in a prehospital setting. Krippendorff's α was 0.60, suggesting poor agreement when triage levels were treated as ordinal data. Full agreement was observed in three case vignettes, whereas agreement across the remaining cases ranged from 32% to 94%. Manual up-triage occurred in 25 of the 30 vignettes, with frequencies ranging from 6% to 74%. No associations were found between assigned triage level and gender, years of EMS experience or specialist education. CONCLUSION: This modified version of SATS demonstrated low inter-rater reliability when applied in a prehospital setting. Agreement varied substantially across patient scenarios, particularly in cases characterized by nonspecific symptoms and near-normal vital signs. These findings underscore the importance of systematic evaluation of modified triage systems to support consistent decision-making and patient safety in prehospital care.