A Framework for Mixed Reality-supported Training of Conflict Resolution and First Responder Skills in International Crisis Situations: SmartSkills

Military first responders and also civilian experts in international peace missions must act quickly and unerringly in complex and dangerous situations under stress. Basic skills must be trained in advance as realistically as possible in order to be able to use them efficiently in later, real-life deployment situations and to reduce the risk potential when deployed to crisis areas. Simulations of concrete conflict scenarios are applied in the biannual “Native Challenge” workshop within a military camp area in the Austrian Alpes that is based on the idea of a cooperation between the “UNESCO Chair for Peace Studies” and the Military Command Tyrol. The representation of these simulations where participants experience in exercise scenarios the surprises, conflicts and dangers they can be confronted within a real mission is very costly which prohibits repeated, personalised and focused skill training.The Austrian project SmartSkills aims at providing standardised scenarios from the most diverse areas (behaviour at the checkpoint, negotiation, accident in the minefield, care of the wounded, etc.) interactively, tailored to the participants in terms of stress load, leadership ability and communication behaviour. A Mixed Reality training system allows unlimited repetitions, especially with personalised difficulty adaptations of the scenarios, and provides corresponding content for in-depth debriefings. SmartSkills will offer a highly innovative automated digital analysis of the human factors of the assignment, in particular the decisive situational awareness in critical situations, and uses biosensors to point out cognitive-emotional problem areas that require special attention in skill development. The associated Decision Support System will translate scientifically validated data into pragmatic risk estimates for the attention of the training management.This presentation will for the firstly describe the conceptual outline of the “Native Challenge” which integrates several levels and feedback loops of operational and strategical challenges. Secondly, the setting and the results of the initial requirement study detailed including information about the selected scenarios and use cases, with roles, objects of interest and training objectives. Furthermore, we will describe the technical specification for the Mixed reality system, its general system architecture, with the outline of wearable bio-signal sensors for psychophysiological monitoring, the graphical processing including realistic object visualisation and digital twins that were scanned from the real operational sites. We will provide details about the study plan, its research hypotheses, in particular, in the context of the evaluation of the training and psychophysiological key performance indicators. Finally, we will describe the anchoring in the international context and the acceleration of developments through exchange with European initiatives.SmartSkills is researching a new dimension of particularly realistic visualisations through the use of innovative digital twins: with highly accurate measurement technology and AI-supported evaluation software, internationally relevant environments can be experienced directly in the simulation centre, thereby increasing the realistic immersion.