Initial evaluation of an AI‐augmented progress monitoring and outcome assessment

Abstract Purpose Most mental health providers have yet to adopt progress monitoring and outcome assessment (PMOA) measures. Although a variety of explanations have been proposed in the literature, a key reason is the burden of time and effort necessary for clients and clinicians to complete, interpret and apply the results of PMOA measures. This evaluation explores the feasibility and initial results of employing ChatGPT to analyse clinicians' unstructured session progress notes for PMOA. Methods Using a simulated patient with 17 trainee therapists, the study examined whether artificial intelligence (AI) can assist in generating thematic summaries relevant to clinical progress and outcomes. Therapists' session summaries were combined to evaluate the continuation of key clinical themes across four sessions for a simulated patient. Trainees also provided brief quantitative ratings per session about the patient's working alliance, negative affect (NA), avoidance of NA and levels of distress. Results AI‐generated results found a (a) persistent focus across sessions regarding the patient's relationship issues with an abusive caretaker, reluctance to disclose and avoidance of NA, and (b) substantial convergence between human‐generated and AI‐generated thematic summaries. Discussion Overall, the use of AI to analyse clinical progress notes appears feasible and psychometrically sound. By minimising resources needed by patients and clinicians to produce clinically relevant data, an AI‐augmented approach can reduce a major obstacle to clinicians' adoption of PMOA measures for feedback purposes.