THEIA-XR: Concept of Virtual User Interface (First Version) (D4.5)

This deliverable is a demonstrator (DEM) presentation. The deliverable. presents the concepts of virtual user interfaces and the state of simulation solutions that are developed and used in task T4.5 Simulation based interaction in which the goal is to utilize simulation and simulator as a platform to evaluate new user interface solutions. The selected interaction concepts identified in the project are implemented in a simulated environment. The internal states of the simulated models are used to simulate sensor data based on which interaction concepts are controlled. The focus is on user interactions with the simulated machine operating in a virtual environment. There are dedicated simulators for each of the use case. The UC1 and UC3 simulators are developed by Creanex and both have the capabilities of the Creanex simulator platform. The UC2 simulator is developed by VTT. Game engine graphics are used for visualization and to test XR visualization features. Physics simulation for the work machines of the use cases, snow groomer, reach stacker and excavator, are implemented to enable development, evaluation and testing in virtual environments. As examples of existing virtualization solutions, a couple of the features of the Creanex simulator platform are presented in section 2. Then an introductory description of the use case simulators is presented in section 3 including description of their main capabilities. Examples of visualization technics and simulation solutions for each use case are presented in section 4 including concept ideas for virtual feedback visualizations. In the deliverable D2.2 Technology analysis and requirements specification (first version) scenarios of the use cases are described including specifications where XR technologies can improve the workflow. The virtual feedback visualizations are based on scenario requirements and identified interaction between the mobile machinery and the human operator. The concepts of the feedback visualizations, which are foreseen to be added to simulations are presented for each use case. This is the first version of the deliverable Concept of Virtual User Interface. The simulated functions and information design of the virtual user interface and simulators are subject to change and will be shaped by the ongoing transdisciplinary co-design process (tasks T3.2 and T3.3). The results of T3.2 will contribute to the development and design of the simulators now that the technological frameworks are in place. The simulations developed for the use cases will also provide a platform to integrate concepts developed in WP3-WP5 into the virtual environment for early testing and demonstration in WP6.