Immersive Fire Training with Live Expert Support and Embodied Interaction with Real Extintor

In this paper, we developed an immersive training environment for mastering the use of a fire extinguisher, with the possibility of receiving live feedback from a remote expert. To facilitate communication and knowledge sharing between trainees and remote experts, we provide bidirectional audio and event synchronization. As additional novel features, we allow the user to see and interact with their own body and real fire extinguisher using deep learning and color-based semantic segmentation. We also allow the expert to monitor the trainee by rendering a 2D video of their egocentric view. Preliminary results show the benefit of receiving live feedback and the potential of using a real fire extinguisher.