Fairness analysis of machine learning predictions of aggression in acute psychiatric care

Machine learning (ML) is increasingly being developed to support individualized risk assessment and de-escalation in acute psychiatry. However, ML algorithms have been shown to exhibit unfair behavior based on protected characteristics, such as an individual's sex or ethnicity. The fairness of ML-based predictions of aggression in acute psychiatry has received limited investigation. To address this gap, we trained an ML algorithm to predict aggressive incidents from structured electronic health records corresponding to 17,703 patients at a large psychiatric hospital between January 2016 and May 2022 (n = 42,719 observation days). We analyzed predictions for fairness by assessing disparities in false positive rates (FPR) and true positive rates (TPR), based on patient race/ethnicity, gender, admission mode, citizenship, and housing status, as well as intersections of race/ethnicity and gender. A random forest algorithm attained ROC-AUC = 0.81. Fairness analyses revealed significant disparities in FPR and TPR across subgroups: FPR was higher for Middle Eastern and Black patients, men, those admitted into emergency care by the police, and those with unstable or supportive forms of housing. Our analysis demonstrates the potential for ML algorithms to reinforce and amplify known social and structural inequities, highlighting the importance of considering and addressing model fairness prior to clinical implementation.