Optimized stacked deep ensemble classifier using hyperparameter tuning for facial expression recognition

Despite significant advancements in FER methodologies, real-time implementation remains challenging due to intra-class variations such as pose, illumination, age, and skin color. These variations cause Single Convolutional Neural Network (CNN) architectures to suffer from high variance, negatively impacting system performance. To address these challenges, this study introduces a method to mitigate intra-class variations by integrating multiple CNN architectures and evaluating them on the FER-2013 dataset. The objective is to develop a Stacked Deep Ensemble Classifier (SDEC) that selects the best-performing models for ensemble learning, reducing variance and enhancing accuracy. The approach leverages both stacking and voting ensemble techniques to improve prediction reliability. Four high-performing base models were designed and developed for this work. In the stacking ensemble, the outputs of these base models were combined and processed through a meta-classifier to generate the final prediction. In the voting ensemble, each model independently predicted class probabilities, which were then aggregated, with the final class determined using the argmax function. The proposed models were evaluated on the FER-2013, achieving a test accuracy of 70.5% for the stacking ensemble and 70.4% for the voting ensemble.