Navigating extreme class imbalance in suicide risk prediction

A low precision for estimated suicide risk can be understood as a consequence of some tradeoffs during model development, particularly training models with matched cases, balanced classes or within short time horizons. This work demonstrates the improved AUPRC performance of regression models in a cross validated framework when these conditions are made more realistic, in the context of class imbalance or less restrictive in that of time horizon. Additionally, we illustrate using the same data that training models in certain clinical cohorts (e.g., defined by age, care utilization and social need) can lead to robustly different estimates for precision and recall, but not AUROC.