Electronic measurement of compliance with a Surgical Care Improvement Program (SCIP) antimicrobial prophylaxis guidelines in the Veterans health administration

Abstract Objective: The Surgical Care Improvement Project (SCIP) was developed and implemented nationally from 2005 to 2015 partly to improve surgical antimicrobial prophylaxis processes. SCIP’s manual review process was costly yet led to high compliance rates in the Veterans Administration (VA). Our objective was to develop an electronic compliance algorithm to replicate the SCIP metric for timely discontinuation of antimicrobial prophylaxis (INF-3) in cardiac surgeries. Study design: This retrospective cohort study matched manually adjudicated INF-3 data from the VA External Peer Review Program (EPRP) with 2006–2015 cardiac surgeries in the Corporate Data Warehouse (CDW). Algorithm development used EPRP data as the gold standard, supplemented with manual review, to apply SCIP-defined electronic exclusions and flag cases with appropriate antimicrobial discontinuation. Algorithm validation used manual review of compliant (n = 90) and non-compliant cases (n = 210) among cardiac surgeries not in EPRP. Results: The algorithm utilized CDW prophylactic versus therapeutic antimicrobial drug names and timestamps for drug administration to reliably exclude cases with appropriate antimicrobial use before flagging INF-3 compliance/non-compliance. In the EPRP-CDW development sample (n = 29,431), the algorithm-measured INF-3 compliance rate was 96.2% versus EPRP’s rate of 95.9%. In validation (n = 10,131), sensitivity was 92.1% (95%CI: 88.0%, 96.2%) and specificity was 97.0% (95%CI: 93.6%, 100%) compared to manual review. The algorithm exclusion rate in CDW cardiac surgeries was 18.5%—primarily for antimicrobial pharmacy administration indicative of pre-or postoperative infection (7,360/52,310 14.1%). Conclusions: The SCIP INF-3 cardiac electronic algorithm effectively replicated the EPRP manual review process and has the potential to extend surveillance of appropriate surgical antimicrobial prophylaxis.