Manually Abstracted vs Electronic Health Record Data for Risk Adjusting Hospital Postoperative Mortality

Importance National surgical quality improvement (QI) programs rely on manual data abstraction to produce hospital-level risk-adjusted outcomes, and while electronic health records (EHRs) could provide an efficient platform for automating data collection of model covariates, the robustness of this approach has not been explored. Objective To evaluate whether risk adjustment using EHR data performs similarly to manually abstracted model covariates for characterizing hospital-level 30-day postoperative mortality. Design, Setting, and Participants This is a national, hospital-level comparative effectiveness study using data from the Veterans Affairs Surgical Quality Improvement Program (VASQIP) and the VA Corporate Data Warehouse (CDW, a repository of national VA EHR data) to compare risk adjustment strategies within a national surgical QI program. VASQIP records were matched to patients’ corresponding CDW record. VASQIP variable correlates were created using VA EHR data. Patients aged 18 years or older who underwent a noncardiac operation at a VA facility from January 1, 2016, to September 30, 2020, with a record in VASQIP were included. Data were analyzed from April 2025 to September 2025. Main Outcomes and Measures The primary outcome was 30-day postoperative mortality. Quarterly observed to expected (OE) ratios were calculated using either EHR or manually abstracted (ie, VASQIP) variables as model covariates and compared to ascertain whether hospital performance was concordantly identified using these different risk adjustment strategies (ie, risk adjustment performed with the same variables but from different data sources). Results Among 386 157 unique patients who underwent 460 585 noncardiac operations at 113 VA facilities, mean (SD) age was 62.9 (13.4) years and 43 781 patients (9.5%) were female. Correlation in OE ratios was strong ( r = 0.98; 95% CI, 0.98-0.98). Across all quarters of data (n = 2111), 2096 hospital quarters (99.3%) were concordantly identified. Among quarters in which hospitals were identified as having higher-than-expected mortality after risk adjustment with VASQIP data (n = 42), 34 (81.0%) were concordantly identified after risk adjustment with parsimonious EHR data. Conclusions and Relevance In this comparative effectiveness study, risk adjustment with EHR data was shown to perform similarly to manually abstracted data for characterizing hospital performance in 30-day postoperative mortality. The use of EHR data for risk adjustment could facilitate automation of data extraction, increase programmatic efficiency, and improve the timeliness of data provision to stakeholders for QI efforts.