AUTOMATIC SCORING OF KNOWLEDGE GAINED AND SHARED THROUGH DISCUSSION FORUMS: BASED ON THE COMMUNITY OF INQUIRY MODEL

The Community of Inquiry (CoI) framework has been widely employed for the past two decades to assess the knowledge gained and shared through online discussion forums. The cognitive presence component of the CoI framework helps identify the evidence of thoughtful knowledge reconstructions through meaning-making during inquiry-based learning. Identifying and scoring these cognitive presences is essential for assessing the students’ learning achievements through online discussion forums. Considering the difficulties associated with manual coding and identifying cognitive presences in discussion forums and the limitations in the existing techniques for auto-identifying and scoring cognitive presences, this research attempted to develop a more efficient tool to identify and score cognitive presences in online discussion forums. The research employed the constructive research approach. The methodology integrated Random Forest (RF) classification with TF-IDF feature extraction and Support Vector Machine (SVM) classification with Word2Vec embedding. A rule-based classifier, constructed upon indicator mappings, enriched the classification process. A weighted voting ensemble method was employed to combine the outputs of the individual classifiers. Our approach was trained and tested on two datasets comprising 781 messages containing 47,592 words. This ensemble method demonstrated notable efficacy, achieving a 69% accuracy rate in classification tasks. This highlights the robustness of the combined approach in enhancing classification performance. Furthermore, the study introduces a scoring model that calculates individual student scores based on post categories, enabling detailed evaluations of student engagement and participation. By assigning scores reflective of discussion contributions, this model advances comprehensive assessments of online learning interactions. Our work contributes to the ongoing conversation on leveraging machine learning for cognitive analysis in online learning environments, highlighting the importance of context-specific methodologies in advancing educational assessment practices.