Advancing Facial Expression Recognition: A Comparative Study of CNNs and Transformers

Over an extended period, facial expression recognition (FER) has been an essential topic in deep learning, with many different models being proposed. Among them, models based on CNN and Transformer architectures are the mainstream choices for FER tasks. This paper compares the performance of mainstream models based on CNN and Transformer on the FER-2013 dataset. The typical representatives of CNN-based models, ResNet50 and VGG16, and the typical representative of Transformer-based models, Vision Transformer (ViT), will be used to learn how to perform FER through transfer learning. They will use a standard and universal training method to ensure the relative fairness of the results. To compare their predictive capabilities, their overall and category-specific accuracy on the test set will be the primary metrics for measuring their predictive abilities. Through comparison, this paper finds that the Transformer-based model, ViT, outperforms ResNet50 and VGG16 in prediction accuracy in more than half of the categories and has the highest overall prediction accuracy, suggesting that Transformer-based models have great potential in the field of FER. The paper is of great reference value for the future mainstream development direction of the FER field.