Automated sentiment analysis in psychotherapy: Convergence with human-coded affective behavior and dyadic associations with early therapeutic alliance

OBJECTIVE: Emotional sentiment expressed during sessions is important in understanding psychotherapy processes and outcomes. Although human coding is typically considered the gold standard for assessing emotions, it is resource-intensive and difficult to scale. Automated sentiment analyses offer promising alternatives, yet little is known about (1) their convergence with human-coded affective behavior at the turn-level rather than session averages, (2) whether the association between automated sentiment and human-coded affect differs by the speaker (therapist vs. client), and (3) automated sentiment indicators' dyadic associations with early therapeutic alliance. METHOD: Using 108 individual early therapy sessions, trained coders coded affective behavior in sessions to generate turn-level positive, negative, and neutral affect scores. Additionally, to automatically quantify sentiment in transcripts at scale, we used a dictionary-based method (LIWC) and a natural language processing-based model (XLM-T). RESULTS: Automated models showed nearly no convergence with human affect coding at the turn-level but limited convergence at the session level. Associations between automated sentiment and human-coded affect for negativity were stronger in clients' than therapists' turns. Automated sentiment was dyadically associated with therapist- and client-reported early alliance. CONCLUSION: Automated sentiment may provide a scalable additional source of information for understanding the therapeutic process and therapeutic relationships.