Semantic labeling of Cognitive Visual Streams Based on learner Behavior Analysis Using Deep-Emotion-AI for Engaging Experience

Consumer Electronics (CE) devices (such as smartphones, tablets, laptops, and cameras) equip learners with educational ubiquity, eliminating barriers of cost, time, and space. The features of emotion recognition and analysis in CE devices have the potential to effectively engage learners through personalized and adaptive recommen- dations of learning contents. Engagement activities triggered by learner’s emotions and expressions rely on the semantic features of facial coding units. Current research integrates Deep-Emotion-AI for learner’s behavior analysis and semantic labeling of emotions. Hence, the system can dynamically gain deeper insights into learners’ emo- tional states and cognitive responses to maximize the learner’s engagement based on real-time streams of facial visuals. Vision-based CE gadgets were employed with five major modules and 68 recognized points of interest in each image of 360x360 dimen- sions. Deep-Emotion-AI utilizes a hybrid of Support Vector Machine (SVM) and Con- volutional Neural Networks (CNN) to recognize learners, capture their expressions, and semantically label learner’s emotions (annotated through Emotions Ontology) over Extended Cohn-Kanade (CK+), MMI and Google datasets. The comparison of CK+ dataset with Google-dataset and MMI highlighted better performance of the proposed model with CK+. An overall accuracy of 98 % was achieved in evaluating semantics and correctly labeling the learner’s emotions. Proposed research has established that the current application of CE with fusion of semantics, deep-emotion-AI and selected feature set can significantly captivate learner’s engagement. The potential future di- rections are deep-learning-based gesture analysis, reinforcing the content precision on learner’s feedback, and efficiently catering for the computational requirements of a larger audience.