Artificial intelligence-based incident analysis and learning system to enhance patient safety and improve treatment quality

Patient safety and high treatment quality are essential in modern healthcare, but analyzing safety incidents for insights is labor-intensive and inconsistent process. To address this, we developed the Artificial Intelligence-based Incident Analysis and Learning System (AI-ILS), trained on 1548 expertly curated incidents categorized by the Human Factors Analysis and Classification System (HFACS). AI-ILS identifies latent safety threats and classifies incident causes with high accuracy, achieving an average AUROC of 0.92, MCC of 0.72, and overall accuracy of 79%. In testing on 350 real-world clinical incidents, AI-ILS showed 88% concordance with expert reviewers and operated 29 times faster than manual analysis. We deployed and validated AI-ILS using real-world radiation oncology data, where it improved retrospective incident analysis at our institution by generating aggregated HFACS-based results and addressing challenges related to inconsistent review processes and lack of standardized taxonomies.