Using Automated Coding of Nonverbal Behavior During a Suicide Assessment to Inform Risk Detection: Mixed Cross-Sectional and Exploratory Prospective Study

Abstract Background Suicide assessments have historically privileged verbal report by the patient, despite the fact that nonverbal behaviors of patients and their clinicians may convey important affective and interpersonal information about suicide risk. Recent advances in computational science enable efficient characterization of rich nonverbal data. Objective This study aimed to use automated coding to test whether facial action and head motion exhibited by young adults and their clinical interviewers during a widely used suicide assessment can identify suicidal participants. Methods Participants were a diverse sample of 66 young adults (age: mean 21.32, SD 2.11 years) recruited from the community, half of whom engaged in past-year suicidal behavior (ie, suicidal participants) and half of whom had no history of suicidality (ie, nonsuicidal participants). Facial action units, head pose, and eye and mouth opening of both participants and clinical interviewers were extracted from the first 3 minutes of a face-to-face, video-recorded Columbia-Suicide Severity Rating Scale (C-SSRS) using the Python-Based Automated Facial Affect Recognition (PyAFAR) software. Nonverbal behaviors of suicidal versus nonsuicidal participants and their interviewers were compared using 2-tailed independent samples t tests and Mann-Whitney U tests. Binary classification algorithms were then used to test how well these nonverbal behaviors together predicted group membership. Exploratory post hoc analyses assessed whether any nonverbal behaviors at baseline were associated with suicidal participants’ ideation severity or suicidal behavior 3 months later. Results Nonverbal behaviors of participants and particularly their clinical interviewers differentiated suicidal versus nonsuicidal young adults at baseline. Suicidal participants demonstrated elevated velocity in opening and closing of eyes and mouth ( P =.004). Interviewers of suicidal participants showed less animated head movement ( P =.02), elevated velocity of eye opening and closing ( P =.02), and ambivalent smiling patterns ( P s=.01-.046). Overall, interviewer nonverbal behaviors predicted group membership with greater accuracy than participant behaviors, correctly identifying 81% (27/33) versus 59% (19/33) of suicidal young adults. Finally, interviewer smiling occurrence was associated with suicidal participants’ ideation severity ( P =.04) and suicidal behavior ( P =.02) 3 months later, while explicit measures, including interviewers’ clinical ratings and participants’ own self-reported ideation severity at baseline, were not. Conclusions This study demonstrates the importance of attending to nonverbal channels of communication in suicide assessments, especially those of the clinical interviewer. It also highlights the potential for automated coding to detect clinically meaningful information efficiently and objectively.