GPT models classify free text from emergency department accesses in children with medical complexity

Abstract Background Emergency Department (ED) management software often collects clinical information as free text, but its unstructured nature limits routine use. While Machine Learning (ML) techniques have been employed to classify such information, they demand substantial data and computational resources. GPT-based models, using direct text prompts, offer a solution without requiring retraining, potentially simplifying clinical practice. Aims To assess OpenAI GPT’s performance in automating data extraction from ED medical records of complex pediatric patients. Methods Electronic admission records of children and young adults referring to the pediatric ED of Padova University Hospital (2007-2023) were analyzed. Principal variables of interest were discharge diagnosis, reason for ED access, required medical procedures, color code at triage, color code at discharge/hospitalization, outcome of the access. GPT-4 classification, via OpenAI API, was compared with manual classification by an expert pediatrician on sample of 791 records. Results Preliminary results of this study analyzed 107 records, showing a correct classification of discharge diagnosis in 95% of records. The reason for access to the ED was correctly identified in 97% of cases, while outcome of admission and color code, both at triage and discharge/hospitalization, were correctly classified in all given cases. Misclassification occurred mainly in cases where classification made by the model was incomplete (e.g., in the case of a trauma, abbreviated site of lesion was not identified by the model). Conclusions GPT-based models demonstrate feasibility, accuracy, and efficiency in transforming unstructured data into structured formats. Ultimately, GPT-based models could provide aid to the medical personnel in decision-making processes. Key messages • Emergency departments often collect clinical data in unstructured free text format, resulting in usability limitations. • GPT-powered models efficiently convert unstructured data to structured formats, offering potential support for medical decision-making.