Quantifying improvement of psychotic symptoms in clozapine-treated schizophrenia: clinical note analysis with large language models

Symptoms of schizophrenia are often reflected in patients' speech. Natural language processing (NLP) approaches enable quantitative assessment of language-related symptoms in schizophrenia. Previous applications have primarily focused on acute psychopathology or predicting the onset or relapse of psychosis rather than treatment-related improvements. Although electronic health records (EHRs) contain rich longitudinal data, unstructured notes hinder structured quantifications. We applied recent large language models (LLMs) to evaluate symptoms based on speech content recorded in EHRs. We analyzed 5,275 clinical notes from 30 patients with treatment-resistant schizophrenia undergoing clozapine treatment. Three state-of-the-art LLMs rated according to the Brief Psychiatric Rating Scale (BPRS). Complementary analysis included parts-of-speech (POS), bag-of-words (BoW), bigram and Linguistic Inquiry and Word Count (LIWC) analyses. LLM-based BPRS ratings revealed significant decreases in Anxiety, Conceptual Disorganization, Suspiciousness, Unusual Thought Content, Hallucinatory behavior, and Depressive Mood during clozapine treatment. POS analysis indicated an increased use of adjectives per sentence, while LIWC analysis revealed more positive emotional expressions during the later phase of treatment. These findings demonstrate that LLMs can extract clinically meaningful symptom information from unstructured clinical text and capture treatment-related changes in psychosis. This approach premises a low-burden method for supporting clinical judgment using routinely collected EHR data.