Synergizing Bernoulli and Entropy for Robust Multi-Scale LBP Image Fusion in Facial Emotion Recognition

Purpose Automatic facial expression recognition systems have advanced significantly in recent years. However, recognizing facial expressions accurately from images remains challenging due to variations in facial characteristics and subtle differences between emotions. Identifying identical emotions may exhibit distinct facial characteristics due to inter-person variability, complicating emotion recognition models. Additionally, discerning between two different facial emotions with minimal variation presents a significant challenge. Despite these challenges, convolutional neural networks (CNNs) have made remarkable progress in advancing emotion recognition. Method Our proposed method aims to address these challenges by detecting facial emotions in two phases. Firstly, we utilize mL-mB fused (One-over-another) Local Binary Pattern (LBP) images with global and local contrast enhancement, along with fusion of global with local contrast enhanced images using Bernoulli's distribution and Shannon entropy. Subsequently, we employ a Customized Convolution Neural Network model (CCNN) to train and recognize facial expressions. Results Experimental findings indicate that our CCNN model achieves significantly improved accuracy when trained using the fusion of global with local contrast enhanced images using Bernoulli's distribution. Notably, the accuracy rates for the CK+, JAFFE, KDEF, FER 2013, and RAF-DB datasets using fused images with global and local contrast enhancement using Bernoulli's distribution are 99.6%, 91.8%, 90.2%, 76.2%, and 49.5%, respectively. These results underscore the effectiveness of our proposed method in facial expression recognition.