Performance fairness of neural network models in early risk assessment of inpatients with varying severity: a retrospective study

To evaluate the performance fairness of a long short-term memory (LSTM) model in predicting in-hospital mortality for inpatients with varying severity, as reflected by length of stay (LOS) and initial clinical scores. This retrospective study used the Medical Information Mart for Intensive Care (MIMIC)-IV database, which includes records from over 50,000 ICU patients. Patients were divided into subgroups based on LOS and Simplified Acute Physiology Score (SAPS) II. The LSTM model was trained on the training set and then tested on these subgroups in the test set. Metrics such as area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), accuracy, sensitivity, and specificity were used to evaluate model performance. Statistical analyses, including logistic regression and Bonferroni correction, were conducted to compare performance across subgroups. The LSTM model’s performance varied significantly among different LOS and SAPS II score groups. The overall AUROC was 0.834, but the model performed better for patients with shorter LOS and lower SAPS II scores. The highest AUROC of 0.931 was observed in the [12, 94) hours LOS group, and 0.811 in the [0, 25) SAPS II score group. The model’s accuracy decreased with increasing LOS and SAPS II scores. Logistic regression confirmed that LOS and SAPS II scores significantly affected model accuracy, with longer LOS and higher SAPS II scores associated with poorer model performance. When using long-term outcomes like in-hospital death to build early assessment models, there are significant fairness issues in model performance across LOS and SAPS II groups. Developing dynamic prediction models using short-term outcomes may help reduce these fairness issues.