Internal Validation of an AI-Assisted Algorithm for PC-06.0 (Unexpected Complications in Term Newborns) Measures

Introduction: The evaluation of perinatal care quality is crucial for ensuring optimal outcomes for newborns. The PC-06 measure, introduced by The Joint Commission in 2019, assesses unexpected complications in term newborns using electronic clinical quality measures (eCQMs) derived from Electronic Health Records (EHRs). Accurate data collection and analysis are vital for healthcare providers to make informed decisions and improve care practices. The PC-06 measure evaluates the outcomes of term newborns using eCQMs derived from EHR data. This process involves the electronic collection and standardized extraction of relevant newborn data. The overall PC-06.0 rate includes both severe and moderate outcomes. Since 2019, our hospital system has tracked this measure, initially through chart review and more recently through EHR-derived eCQMs. The Obstetric Safety Committee observed an increase in the PC-06.0 rate following the implementation of the algorithm. This study aims to internally validate the eCQM tool used to measure PC-06.0 (Unexpected Complications in Term Newborns) in a large referral-based community hospital. Study Design: This retrospective chart review of 2023 delivery data aligns with the PC-06.0 measure at a large referral-based community hospital in Northwest Ohio. Neonates born at and remaining in the hospital were identified and classified under the PC-06.0 measure using the algorithm. A diverse group of reviewers was selected to review neonate charts and validate the categorization of newborns under PC-06.0. Results: The initial PC-06.0 rate for 2023 was 29.29%, with 104 newborns identified as having unexpected complications in the term newborns. After validation, only 86 (82.69%) of these cases were confirmed, resulting in a revised PC-06.0 rate of 21.1% for deliveries. Conclusion: Our findings suggest that the algorithm used for identifying PC-06.0 overestimated the number of unexpected complications in term neonates, leading to inflated PC-06.0 data and reports. Providers should exercise caution when relying on algorithms and artificial intelligence for data collection, as these tools can potentially result in inaccurate data. This study underscores the importance of rigorous validation processes to ensure the accuracy of perinatal care performance metrics.