Deep learning-based natural language processing for critical care identification in pediatric emergency department

BACKGROUND: The initial assessment of pediatric emergency patients is challenging due to diverse clinical presentations and the limited applicability of existing mortality scoring systems that require detailed laboratory results often unavailable at first encounter. Deep learning (DL)-based Natural Language Processing (NLP) offers the ability to extract complex patterns within textual data attained at the first encounter with the physician and can enhance the accuracy and timely prediction of critically ill patients in the Pediatric emergency department (PED). This study identified whether DL-based NLP techniques, which integrate structured data (vital signs) with unstructured data (clinical text written by physicians), can accurately identify patients requiring critical care in PEDs. METHODS: Four machine learning (ML) models using term frequency-inverse document frequency-logistic regression, extreme gradient boosting, gradient boosting, and random forest, and two DL-based NLP models based on the Korean Medical Bidirectional Encoder Representations from Transformers (KM-BERT) framework were tested on 87,748 children, including 599 (0.7%) critical cases and 14,095 (16.1%) admission cases in a tertiary hospital, from January 1, 2012, to December 31, 2021. RESULTS: KM-BERT, using a Masked Language Model (MLM), outperformed all ML-based models in predicting critically ill patients, achieving the highest Area Under the Receiver Operating Curve (AUROC) of 0.941 ± 0.015 and an accuracy of 0.952 ± 0.002. It also demonstrated superior performance in predicting hospitalized patients, with an AUROC of 0.879 ± 0.004 and an accuracy of 0.798 ± 0.002. CONCLUSION: These findings suggest that DL-based NLP approaches, particularly KM-BERT with MLM, may improve the early identification of critically ill and hospitalized pediatric patients in the PED compared with conventional ML-based models. Further prospective, multi-center studies are required to validate these findings and establish their suitability for routine clinical integration.