Emotion Recognition from Human Speech Using Linguistic and Paralinguistic Features Through a Soft Voting Approach

Emotions are intricate psychological and physiological reactions that arise in response to various triggers and influence how people interact with others, face challenges, and make decisions. When it comes to interpersonal communication, a person's emotions are typically conveyed through both verbal and nonverbal cues. Speech serves as an effective signal for recognizing emotions as it encompasses both linguistic and paralinguistic information. This study proposes an emotion recognition model that utilizes speech to identify emotions. The linguistic aspects of speech, which involve the contextual meaning of an utterance including the arrangement and choice of words, are considered. Additionally, the paralinguistic aspects, such as pitch and intensity, which influence a person's intonation and speech prosody, are included. The model comprises two subsystems, each separately trained. The probability weights from these subsystems are calculated and used with a soft voting ensemble classifier model to make final predictions based on five emotion classes. Through experiments, the soft voting ensemble approach in the emotion recognition model achieved an average accuracy performance of 81 %, which was significantly higher than the average accuracy performance of the individual approaches for each subsystem.