Study on Multimodal Session Recognition Based on Graph Neural Networks

As one of the special research directions of text emotion recognition task, the conversation emotion recognition task needs to take the conversation context, speaker and other factors into account to accurately identify the emotion state.To address challenges in multimodal information fusion and global-local feature extraction, this paper proposes a multimodal session recognition method based on multilevel attention mechanism and multistream graph neural network.Firstly, two key issues of the multimodal session recognition problem are analysed; secondly, a multimodal session emotion recognition model based on the improved attention mechanism strategy and the improved graph neural network is proposed to address the two issues; and finally, the effectiveness of the proposed method is verified through the analysis of simulation experiments.The simulation results show that the method achieves good accuracy in IEMOCAP and MOSEI datasets and effectively improves the efficiency of session fusion recognition.