Measuring workflow interruptions in endoscopy: development and interrater reliability of a structured observation tool

Background Workflow interruptions are common in procedural medicine and may increase cognitive load, stress, and the risk of errors. While interruptions and flow disruptions have been studied in surgical settings, little is known about workflow interruptions in endoscopic environments. Objective evaluation of workflow interruptions requires a reliable measurement tool. This study aimed to develop a structured observation tool for recording workflow interruptions during endoscopic procedures and to evaluate its interrater reliability. Methods A structured tally-based observation tool for recording workflow interruptions was developed in an iterative process, and interruptions were categorized into predefined domains. Three independent observers underwent structured training and calibration. A predefined subset of endoscopic procedures was observed simultaneously by two independent observers. Interrater reliability was assessed using intraclass correlation coefficients (ICC) and Bland–Altman analysis. Results A total of 35 procedures were scheduled for double observation, and 33 procedures were included in the final interrater reliability analysis. Interrater reliability for the total number of interruptions was excellent for both observer pairs (ICC 0.995 for A-B and 0.995 for A-C). Reliability across interruption domains ranged from good to excellent. Bland–Altman analysis demonstrated small mean differences with narrow limits of agreement and no relevant systematic bias. Conclusion This methodological study demonstrates that workflow interruptions during endoscopic procedures can be measured reliably using a structured tally-based observation tool. The instrument provides a standardized methodological framework for future studies investigating workflow processes and workflow-focused interventions in endoscopy.