Development of an Artificial Intelligence-Based Text Sentiment Analysis System for Evaluating Learning Engagement Levels in STEAM Education

This study aims to create an AI system that analyzes text to evaluate student engagement in STEAM education. It explores how sentiment analysis can measure emotional, cognitive, and behavioral involvement in learning. We developed an AI-based text sentiment analysis system to assess learning engagement, integrating speech recognition, natural language processing techniques, keyword analysis, and text sentiment analysis. The system was designed to evaluate the level of learning engagement effectively. A computational thinking curriculum and study sheets were developed for university students, and students’ participation experiences were collected using these study sheets. The study utilized the strengths of SnowNLP and Jieba, proposing a hybrid model to perform sentiment analysis on students’ learning experiences. We analyzed: 1, The effect of sentiment dictionaries on the model’s accuracy; 2, The accuracy of different models; and 3, Keywords. The results indicated that different sentiment dictionaries had a significant impact on the model’s accuracy. The hybrid model proposed in this study, utilizing the NTUSU sentiment dictionary, outperformed the other four models in effectively analyzing learners’ emotions. Keyword analysis indicated that teaching materials or courses designed to promote practical, fun, and easy ways of thinking and building logic helped students develop positive emotions and enhanced their learning engagement. The most frequently occurring keywords associated with negative emotions were “problem”, “error”, “not”, and “mistake”. This finding suggests that learners experiencing challenges during the learning process—such as encountering mistakes, errors, or unexpected outcomes—are likely to develop negative emotions, which in turn decrease their engagement in learning.