48. Large Language Models for Perceptual Speech Clinical Data Extraction in Cleft Palate Speech Language Pathology Notes

PURPOSE: Speech-language pathology (SLP) assessments are vital for supporting plastic surgeons in monitoring pediatric patients with cleft palate and velopharyngeal insufficiency (VPI) by providing qualitative and quantitive data on a patient’s functional speech status. Traditional manual extraction of speech notes is time-intensive and susceptible to inconsistent subjective interpretation. Recent advancements in natural language processing, specifically large language models (LLMs), offer a potential solution for automating and enhancing the extraction of diagnostic information from clinical texts. This study evaluates the efficacy of an in-house developed version of the GPT-4 model in extracting concepts from speech sample notes written by speech-language pathologists. METHODS: We conducted a retrospective analysis of 5 anonymized speech sample notes, authored by a speech-language pathologist, for patients aged 3-11 years (average: 6.68, SD: 3.54). These notes were processed using an in-house adaptation of the GPT-4 model, termed VERSA, to extract key speech and language concepts. The model was tasked with identifying patient’s diagnosis, surgical history, and scaled elements from each note such as assessments of resonance and nasal airflow. Resonance evaluations included hypernasality, rated on a 0-4 scale, and hyponasality on a 0-2 scale. Nasal airflow evaluations were conducted to assess audible nasal emission or nasal turbulence, rated on a 0-2 scale. Additionally, grimace was rated as 0 or 1, and speech acceptability was assessed on a 0-3 scale. RESULTS: Our model successfully extracted the patient’s diagnosis, any surgical history, and the date of surgery from all analyzed speech sample notes. The VERSA model efficiently identified clinically relevant speech evaluation elements for measures of hypernasality, hyponasality, nasal airflow, and grimace and were accurately extracted in four out of five applicable cases. In one instance, the model indicated “Unable to rate due to lack of verbal output”, correctly recognizing the absence of necessary verbal information for evaluation. CONCLUSION: The application of the VERSA model demonstrates substantial potential in streamlining the extraction of diagnostic and evaluative information from pediatric speech pathology notes These preliminary findings highlight the potential of LLMs to enhance the efficiency and reliability of clinical assessments in speech-language pathology. For clinicians, this model reduces the time spent manually retrieving information and mitigates subjective interpretation of speech data. Future research should focus on expanding the dataset and refining model parameters to further improve accuracy and applicability in diverse clinical scenarios. In the future, we aim our LLM to be able to summarize multiple speech evaluation in sequential order, which would further allow the cleft team to focus on the patient rather than looking up histories.