Automated algorithm in catheter-associated urinary tract infection identification using digital health records: validity and barriers in implementation into clinical practice

This study aims to evaluate the accuracy of an automated algorithm designed to identify catheter-associated urinary tract infections (CAUTIs) using electronic health records from a hospital. We assess the algorithm’s effectiveness as a clinical decision support tool by analyzing its ability to accurately identify CAUTIs based on predefined clinical parameters. We conducted a retrospective analysis of all patients hospitalized in the acute internal medicine wards of the Cantonal Hospital of Baden between November 2022 and October 2024. Automated algorithm identifies potential CAUTI cases based on standard centers for disease control (CDC) criteria. The automated algorithm identified patients meeting these criteria as potential cases and the number of CAUTIs is visualized in a dash board. All records were manually reviewed and inconsistencies validated by an infection specialist. Sensitivity, specificity, area under the receiver operating characteristic (ROC) curve (AUC), likelihood ratios (LR) and 95% confidence intervals were calculated to assess performance. Between November 2022 and October 2024, a cohort of 1424 patients with indwelling catheters was assessed, resulting in the identification of 11 manually diagnosed CAUTIs. The automated algorithm identified 6 of these cases, yielding 6 true positives (TP), 1,385 true negatives (TN), 28 false positives (FP), and 5 false negatives (FN). The algorithm demonstrated a sensitivity of 55% (95% CI: 27.3% − 81.8%) and a specificity of 98% (95% CI: 97.3% – 98.7%), positive likelihood ratio (LR+) of 27.3 and a negative likelihood ratio (LR−) of 0.5. AUC of 0.76 (95% CI: 0.73–0.80) with P-value of 0.0001. In this validation study, the automated algorithm demonstrated high specificity (98%) but limited sensitivity (55%) for the detection of CAUTIs. While a positive algorithm result was strongly associated with true infection, nearly half of confirmed CAUTI cases were not identified. Therefore, the algorithm may support surveillance by prioritizing cases for review; however, due to its limited sensitivity, it cannot reliably exclude non-cases and should not replace manual case adjudication. The primary reasons for missed cases were gaps in electronic health record (EHR) documentation. In hospitals with comprehensive EHR use across all departments, improved sensitivity is expected.