Hidden Emotion Detection using Speech, Text, Facial Expressions and Neural Networks

Understanding and recognition of human emotions are very crucial in various fields. This paper proposes a new approach to show the different feelings that are hidden using multi-modalities like video, audio, and textual data. A custom dataset AED-FST has been used in this work which has nearly 317 samples each consisting of some audio clip, some visual media, and a question-and-answer pair. From all these entirely diverse sources, the goal is to create a model that could possibly detect emotional cues. It found hidden emotions with a 67% accuracy, representing how well it could pick up on these weak cues from all different modalities. This method contributes immensely to emotion detection in extremely wide fields, including psychological diagnoses and human-computer interaction, to mention just a few.