Improving Frame-based Engagement Classification in E-Learning Using EfficientNet and Normalized Loss Weighting

Engagement can be defined as how individuals are involved in and interact with a task that requires attention and emotional conditions. Engagement is an affective state positively correlated with learning processes. Engagement along with other affective states, such as boredom, confusion, and frustration must be analyzed to identify students’ learning behavior. Implementing proper prevention by measuring student engagement levels could increase students’ learning intake. Such implementation involves building an effective feedback system or rearranging the learning design. Several researchers have proposed deep-learning approaches using the DAiSEE dataset to classify student engagement levels. In addition, previous studies utilized various loss functions equipped with class weighting to assign higher importance to the minor classes, which are low and very low engagement classes. Most of the state-of-the-art models achieved high accuracy, but the f1-score was still low because of the minor class struggle. This research tries to solve engagement level classification on imbalance conditions by proposing a normalized loss function weighting based on the Inverse Class Frequency formula based on each class’ instances to give more importance and focus to the classes and trained on Vanilla EfficientNet model rather than experimenting on more advanced model to keep the efficient and suit the memory constraint on the e-learning implementation. Based on the conducted experiments, the normalized ICF obtained the highest accuracy of 51.64% and weighted f1-score of 50.86%, which is superior to the standard ICF performance, which received 50.32% accuracy and weighted f1-score of 50.49% using the same settings.