Assessing Collaboration Using Multimodal Signals in a Triadic Puzzle-Solving Task in Virtual Reality

In high-stakes collaborative situations, a decline in collaboration quality can lead to significant adverse events. Although human factors (HF) specialists are effective in identifying and resolving collaboration issues, their analyses are often case-specific and conducted retrospectively. To address these limitations, our research aims to assess collaboration processes in real-time using multimodal signals collected and analyzed during the activity. In this context, we propose a platform that integrates audio, visual, task logs, and spatial metrics computed along the activity, to provide an assessment of various dimensions of the collaboration process. This approach is applied to a triadic collaborative puzzle-solving task in virtual reality.