Estimation of ICAP States Based on Interaction Data During Collaborative Learning

The primary goal of this study is to investigate a method for estimating the state of learners in the near future using nonverbal information used in multimodal interaction as cues to provide adaptive support in collaborative learning. We used interactive-constructive-active-passive (ICAP) theory to classify learners' states in collaborative learning. We attempted to determine whether a learner’s ICAP state was passive based on multimodal data obtained during a collaborative concept-map task. We conducted an experiment on collaborative learning among learners and acquired data on conversational type, the results of learning performance (pre- and post-tests), utterances, facial expressions, gaze, and voice during the experiment. We conducted two analyses. One was sequential pattern mining, to obtain clues for predicting the participants' state after 5 seconds. The other was a support vector machine to try to classify the participants' state based on the obtained clues. We found several candidates that could be used for learner-state estimation in the near future. The learner-state estimation using multimodal information yielded higher than 70% accuracy. In contrast, there were differences in the ease of estimating each pair's learning state. It appears that capturing the characteristics of interactions in collaborative learning for each pair is necessary for a more accurate estimation of the learners' state.