Identifying Comprehension Fault from Word Occurrences in Writing Questions

This study establishes a method for determining whether learners have a comprehensive understanding of data science. Data science requires knowledge and skills in various fields, which makes many learners give up in the process. To prevent learners from being discouraged, it is necessary to judge the comprehension of the principles in each specified skill. Writing questions are effective in examining comprehension. However, difficulties arise when examining comprehension from writing questions. For each specified skill, this method provides us with a method to examine whether learner answers to writing questions are close to the correct answer. The Word2vec method calculates the distance of the document vector of learner answers from that of the answer sentence. The distance discriminates learners lacking comprehension from others in the skill. The method uses the Naïve Bayes Model to figure out important words for each skill. Learners lacking comprehension fail to use the important words in their answers. A logistic regression model to discriminate learners in terms of comprehension reveals the words, which contribute to identifying the concepts they fail to understand and the procedures they fail to associate with the concepts. The experimental results show that the method can determine what the learner has understood. Furthermore, it identifies the important words in classifying learners lacking comprehension from others. The words appearing in answers of learners who understand and those who do not are extremely far in their distributed representation. It suggests that it is possible to identify concepts and procedures that learners do not understand from the words they use. It enables us to offer suggestions to assist learners who are likely to be stuck.