Attention based ConVnet-Recurrent Neural Network for Facial Recognition and Emotion Detection

Facial Emotion Recognition (FER) plays a crucial role in analysing and understanding individual’s emotions in the form of non-verbal communication. Existing Deep Learning (DL) based FER methods offered diverse contribution in analysing human emotions but several challenges like extracting accurate spatial and temporal features for accurate classification of emotions is a major concern. To address this, a DL based Facial Recognition and Emotion Detection (FRED) system is proposed by using extended Cohn-Kanade (CK+) dataset as the input. The data is pre-processed first where the Histogram Equalization (HE) is applied and then Convolutional Neural Network (CNN) is used to extract the spatial features while reducing the dimensionality. The extracted features are given to Recurrent Neural Network (RNN) to learn long term dependencies from a temporal sequence of data. The final classification is performed where the Attention Mechanism (AM) is integrated in the RNN to determine the higher weights for better outcome. The results of proposed AMConVnet-RNN showed a superior classification accuracy of 99.54%, precision of 98.63%, and f1-score of 97.39% when compared to existing FER methods like Residual Network-50 (ResNet-50) and Channel Based Attention Mechanism (CBAM) based ResNet50.