Effect of Attention Block in Learning of Facial Expressions Using Teacher–Student Knowledge Distillation

Facial expression recognition (FER) is a rapidly evolving field of study with various applications in psychology, security, and computer science. Nonverbal cues, such as body language, gestures, and facial expressions, are important in daily life because they can convey more about the feelings and desires of a person. AI systems can enhance relationships and social interactions by recognizing these signals. This study contributes to this field by including a knowledge distillation (KD) process, with teacher models (TM) based on a large pre-trained convolutional neural network (CNN) and student models (SM) consisting of a smaller pre-trained CNN and classifiers. In the KD process, the teacher (TM) and student (SM) models proposed were Res-Net101/ResNet50, EfficientNetB7/EfficientNetB0, and MobileNetV3-large/MobileNetV3-small. The AffectNet and Real-world Affective Faces Database (RAF-DB) datasets were used for general steps learning. In an ablation process, the salient TM and SM were trained on the datasets. The CBAM (Convolutional Block Attention Module), ECA (Efficient Channel Attention), and FcaNet (Frequency Channel Attention Network) attention blocks were inserted after the last convolutional block in both TM and SM, or sequentially, with the aim of refining the deep features and improving the performance of CNNs.

Authors

Institutions

Publication Details

Journal
Knowledge
Published
2026-11-16
DOI
https://doi.org/10.3390/knowledge6040030
Primary Topic
Emotion and Mood Recognition
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Effect of Attention Block in Learning of Facial Expressions Using Teacher–Student Knowledge Distillation

Cătălin Anghel, Sanda Florentina Mihalache, Simona Moldovanu, Ioana Diana Moldovanu
3 links
Knowledge
Emotion and Mood Recognition
article

Effect of Attention Block in Learning of Facial Expressions Using Teacher–Student Knowledge Distillation

Cătălin Anghel, Sanda Florentina Mihalache, Simona Moldovanu, Ioana Diana Moldovanu
article en

Abstract

Facial expression recognition (FER) is a rapidly evolving field of study with various applications in psychology, security, and computer science. Nonverbal cues, such as body language, gestures, and facial expressions, are important in daily life because they can convey more about the feelings and desires of a person. AI systems can enhance relationships and social interactions by recognizing these signals. This study contributes to this field by including a knowledge distillation (KD) process, with teacher models (TM) based on a large pre-trained convolutional neural network (CNN) and student models (SM) consisting of a smaller pre-trained CNN and classifiers. In the KD process, the teacher (TM) and student (SM) models proposed were Res-Net101/ResNet50, EfficientNetB7/EfficientNetB0, and MobileNetV3-large/MobileNetV3-small. The AffectNet and Real-world Affective Faces Database (RAF-DB) datasets were used for general steps learning. In an ablation process, the salient TM and SM were trained on the datasets. The CBAM (Convolutional Block Attention Module), ECA (Efficient Channel Attention), and FcaNet (Frequency Channel Attention Network) attention blocks were inserted after the last convolutional block in both TM and SM, or sequentially, with the aim of refining the deep features and improving the performance of CNNs.

KnowledgeVol. 6(4)
University of Bucharest (RO), "Dunarea de Jos" University of Galati (RO), Petroleum & Gas University of Ploieşti (RO)
Quality education
Emotion and Mood Recognition
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon