Toward multimodal learning analytics in simulation-based collaborative learning: A design ethnography of maritime training

Abstract Collaborative learning in high-fidelity simulators is an important part of how master mariner students are preparing for their future career at sea by becoming part of a ship’s bridge team. This study aims to inform the design of multimodal learning analytics to be used for providing automated feedback to master mariner students engaged in collaborative learning activities in high-fidelity navigation simulators. Through a design ethnographic approach, we analyze video records of everyday training practices at a simulator center in Scandinavia, exploring (a) how feedback is delivered to students during collaborative activities in full-mission simulators and (b) which sensors are needed and why they are needed for capturing the multimodal nature of professional performance, communication, and collaboration in simulation-based collaborative learning. Our detailed analysis of two episodes from the data corpus shows how the delivery of feedback during simulations consists of recurring, multidimensional, and multimodal feedback cycles, comprising instructors’ close monitoring of student’s actions to continuously assess the fit between the learning objectives and the ongoing task. Through these embedded assessments, feedback that draws on the rich semiotic resources of the simulated environment, while considering aspects of realism and authenticity, is provided. Considering the multidimensional and multimodal nature of feedback in professional learning contexts, we identify technologies and sensors needed for capturing professional performance in simulated environments.