1161 Identifying Ethologically Relevant Neurobehavioral Biomarkers of Emotional State

INTRODUCTION: Major depressive disorder (MDD) is the leading cause of disability and death from suicide. Emotion dysregulation is the hallmark of depression, so developing tools for objective, quantitative characterization of the temporal, behavioral, and neural dynamics underlying emotional state is critical for properly diagnosing and treating these debilitating conditions. METHODS: We analyzed continuous, synchronized audio, video, and neural recordings during daily social interactions in human neurosurgical patients implanted with both stereo-EEG and deep brain stimulation (DBS) electrodes as part of a clinical trial (NCT03437928) for treatment-resistant depression (TRD). We then developed a pipeline for automated transcription with diarization and utterance-level timestamps of audio recordings. Pre-trained affective computing models were used for extraction of linguistic, acoustic, and kinesic features associated with emotional state. These behavioral features were then correlated to spatio-spectral features of concurrent neural activity. Finally, we used an intermediate fusion model to investigate the relationship between neural activity and periods of cross-modal convergence. RESULTS: We found a strong relationship between emotional features of patient behavior and neural activity during natural conversations. Specifically, the first canonical component identified a maximal correlation between the valence and arousal dimensions of cross-modal behavior associated with positive affect and a distinct pattern of neural activity characterized by increased high-frequency and decreased low-frequency activity, particularly in limbic regions (amygdala and anterior cingulate cortex). CONCLUSIONS: Naturalistic conversations provide a wealth of objective, quantifiable behavioral data that is highly temporally resolved and closely aligned with underlying neural activity. By relating linguistic features from “what” is expressed, as well as acoustic and kinesic features from “how” it is expressed, to simultaneous neural activity, we can build multi-modal models for more effective diagnosis, assessment, and treatment of affective disorders.