Enhancing Facial Expression Recognition by Self-Distillation

Facial Expression Recognition (FER) is a critical research area with wide-ranging applications. While advanced convolutional neural networks have enhanced FER performance, their complexity impedes deployment on resource-constrained devices. Inspired by the self-distillation architecture, this study introduces ResNet18-Lite, featuring a multi-branch architecture with Distillation Blocks designed in its shallow branchs. This novel design not only improves overall performance on FER tasks but also allows flexible branch selection during prediction, making it an optimal solution for resource-limited FER applications. Experimental results on popular FER datasets, including FER2013 and FER-Plus, demonstrate ResNet18-Lite’s superior performance compared to baseline deep learning models and other recent ResNet18 variants with Self-Distillation. Notably, ResNet18-Lite’s lightest classifier branch, utilizing only 0.9 million parameters, outperforms other more complex models.