Facial Emotion Recognition for Enhanced Human-Computer Interaction using Deep Learning and Temporal Modeling with BiLSTM

Facial Expression Recognition is an essential component in the development of Human-Computer Interaction (HCI) because it enables the system to comprehend and appropriately react to human feelings. This study presents deep learning approaches for FER that are based on the combination of a Convolutional Neural Network and a Bidirectional Long Short-Term Memory Network. These approaches are intended to be utilized for the purpose of utilizing spatial and temporal variables in order to develop robust emotion categorization processes. The JAFFE and CK+ datasets are taken into consideration in the suggested technique. Both sets contain images that pertain to seven fundamental emotion classes, which are as follows: happy, sorrow, surprise, anger, disgust, fear, and neutral. The process of preprocessing begins with the detection of faces through the utilization of Multi-task Cascaded Convolutional Networks (MTCNN), followed by the normalization and scaling of the faces in order to maintain uniformity in the input dimensions. On the other hand, the BiLSTM component is responsible for capturing temporal dynamics that describe face expressions over time, while the CNN component is responsible for extracting spatial characteristics from the images. Considering that both sorts of features are incorporated into a single structure, it is evident that such a combination architecture will result in an improvement in recognition accuracy. Following that, it was trained on the preprocessed datasets for a total of 25 epochs, which resulted in a significant improvement in its accuracy. The fact that the model generalizes effectively to data that it has not before encountered is demonstrated by the fact that the accuracy it achieved during training and validation was $\mathbf{9 5 . 2 \%}$ and $\mathbf{9 3 . 1 \%}$, respectively. The combination of CNN and BiLSTM is able to effectively address the challenges that are associated with FER, such as issues pertaining to illumination, position, and the variable intensity of expressions.