A Learning Assistant Using Facial Expression Recognition

Understanding and interpreting the state of mind of a learner is very essential in creating an effective learning experience for the learner. While the requested information is being displayed for the user, the frames from the webcam are captured at regular intervals to recognize the facial expressions of the user. The expressions are identified, and the reasons behind such expressions are analyzed to modify the presented information in a way that the user might be interested in. A model is trained using deep learning on the FER-2013 dataset to classify each frame into one of the seven classes of facial expressions: happy, surprise, angry, fear, disgust, sad, and neutral. The predominant expression throughout the learning duration is identified, and the given information is adapted based on the predominant expression. The proposed application will aid in giving personalized responses by modifying the presented information in a dynamic manner, leading to an effective learning environment where the state of mind of the user is understood without the use of natural language and suitable modifications to the learning content are provided. The deep learning model used to classify facial expressions has reached an accuracy of 86.51%.