Automated calculation of healthcare quality and safety indicators for head and neck cancers: a multicentric study using electronic health records

To develop and validate algorithms to automate the calculation of Healthcare Quality and Safety Indicators (HQSIs) from electronic health records (EHR) multi-sources data. Three HQSIs of interest were identified by experts. The required variables were identified and algorithms for extracting these variables from various sources of EHR data, prioritized based on their quality, were performed and evaluated on the Assistance Publique - Hôpitaux de Paris (AP-HP) and Bordeaux University Hospital clinical data warehouses (CDWs). Algorithm performances (positive predictive value (PPV), accuracy, f1-score) were assessed to a manual review of 100 EHRs randomly selected among newly referred patients with head and neck cancer (HNC) in 2023. Three HQSI were computed: number of newly referred HNC, number of HNC cases treated by front line anticancer surgery and the number of resected HNC cases requiring a post-operative surgery based on ICD-10 and their French Association pour le Développement de l’Informatique en Cytologie et en Anatomie Pathologique (ADICAP) pathology codes, 6,943 and 7,112 HNC patients were identified as referred to AP-HP and Bordeaux University Hospitals between 1998 and 2023, respectively. The algorithm related to the number of newly referred HNC diagnoses in 2023 had the following performances: PPV of 37% and 87% when ICD-10 codes solely were used, up to 89% and 100% when both data sources (ICD-10 and ADICAP codes) were used, in AP-HP and Bordeaux University Hospitals, respectively. The average discrepancy rate between both data sources was 44%. Based on surgery French coding of medical procedures (CCAM) and pathology ADICAP codes, 4, 231 newly referred HNC patients were identified with a frontline cancer surgery in AP-HP, between 1998 and 2024. The accuracy of such algorithm ranged from 65% for patients identified by CCAM solely to 84% for patients identified by CCAM and ADICAP codes. Among them, 436 patients had a surgical re-intervention in the 28 following days. Floor of the mouth (OR = 1.72; CI95, 1.02–2.90), mouth (OR = 1.59; CI95, 1.07–2.38), pyriform sinus (OR = 4.83; CI95, 1.65–14.13), and facial sinus cases (OR = 0.11; CI95, 0.02–0.82) were primary tumor sites with a modified risk of surgical re-intervention, respectively. Leveraging data from heterogeneous sources of EHR remains promising to improve the HQSI automated measurement for HNC patients.