Adapting Vision Transformers for Effective Facial Expression Recognition

Facial Expression Recognition (FER) is a vital area of research in affective computing, permitting systems to automatically interpret and respond to human emotions by studying and analyzing facial expressions. FER involves recognizing diffused variations in facial features corresponding to various emotional states. This study explores the performance of Vision Transformer (ViT) model on the FER task. Specifically, the vitbase-patch16-224-in21k was fine-tuned on an “in the wild” dataset named AffectNet containing facial images of 8 different emotion classes. The classifier head of the ViT model was modified to fit the FER specifications, and the data preprocessing contained augmentation methods like rotation, flipping, and grayscale-to-RGB conversion to improve model robustness. Testing on the AffectNet-7/8 dataset resulted in accuracies of 62.17% and 57.56%, respectively. The research discusses the capabilities of vision transformers in FER while it also mentions areas of improvement.