Speech Emotion Recognition Based on Adaptive Feature Fusion Network

Speech emotion recognition technology plays an indispensable role in the field of artificial intelligence, which can effectively improve the comfort of human-computer interaction. Different speech signal features can represents emotion state from different levels, and the fusion of those features can improve the performance of speech emotion recognition systems. However, the existing works lacks of effective feature fusion mechanisms, which lead to the poor performance of speech emotion recognition system. In order to address this problem, we propose a novel adaptive feature fusion network (AFFN),which can fully leverage the emotional representation capabilities of different features. In the proposed model, we designed two parallel deep learning feature extraction channels to learn mel-spectrogram and sound feature features, respectively. In the feature fusion stage, an adaptive feature fusion strategy is adopted, which can fully consider the matching performance between different features, efficiently fuse emotional features from different dimensions, and effectively improve the effectiveness of emotion recognition system. Extensive experiments are conducted on the designed AFFN based on IEMOCAP and EMODB databases, and the experimental results show that the proposed method outperforms existing feature fusion schemes in both WA and UA performance.