Knockoff-ML: a knockoff machine learning framework for controlled variable selection and risk stratification in electronic health record data

Effective risk stratification is essential in clinical practice, enabling better resource allocation and improved patient outcomes. Although machine learning models have been widely used for risk prediction and stratification in electronic health record (EHR) data, conventional interpretability metrics for machine learning models typically lack decision rules for clinicians to determine which patient features significantly influence outcomes. We proposed Knockoff-ML, a model-free machine learning framework that simultaneously accomplishes outcome prediction and identification of risk features through integrating a knockoff framework with various predictive machine learning algorithms. Specifically, Knockoff-ML augments traditional machine learning models with the knockoff framework that enables machine learning models to perform variable selection with false discovery rate (FDR) control in the presence of complex, nonlinear associations between features and outcomes in EHR data. We extensively evaluated Knockoff-ML in both simulations and real-data applications. Our simulation results demonstrated that Knockoff-ML consistently achieved high statistical power to identify risk features while rigorously controlling the FDR, whereas conventional feature selection methods exhibited inflated FDR in most scenarios. In applications to a cohort of 50,591 intensive care unit (ICU) patients from the Medical Information Mart for Intensive Care (MIMIC)-IV database, Knockoff-ML identified risk features significantly associated with short- and long-term mortality. Prediction models with identified risk features in Knockoff-ML also achieved comparable prediction accuracy with full models using all available features. Furthermore, Knockoff-ML exhibited substantially higher predictive power and clinical utility compared to conventional ICU scoring systems such as SOFA and SAPS II. The robust performance and interpretability of Knockoff-ML make it a useful tool for enhancing clinical decision-making, with the potential to significantly improve patient outcomes and optimize healthcare delivery.