Modeling Implicit Emotion and User-specific Context for Malevolence Detection in Mental Health Counseling Dialogues

Generative conversational agents, driven by large language models, have gained widespread popularity. However, a significant drawback lies in their tendency to produce uncontrollable and unpredictable contents, thereby increasing the risk of generating malevolent responses that potentially exacerbate users’ mental health issues. Although existing research on malevolence detection in dialogues addressed the modeling of interaction patterns in dialogues, the implicitly expressed emotion and user-specific context are often neglected. Addressing this gap, we propose a hypergraph-enhanced context modeling approach for detecting malevolence in mental health counseling dialogues. Our approach harnesses the emotion reasoning capabilities of large language models to generate implicit emotional prompts. Employing hypergraph neural networks, our approach effectively integrates emotional context, user-specific context, and interactive context, fusing them into high-order semantic representations using hypergraph convolution. Experimental results on two benchmark datasets, MDRDC and Dialogue Safety, demonstrate the superiority of our model over state-of-the-art baseline models, particularly in complex contextual scenarios.