Understanding Human Behavior In Mixed Reality Juggling Simulator With Haptic Interaction

Mixed reality is becoming an integral element of increasing educational and work processes. To create maximum nativeness of the process, users in most cases use their own hands as the main element of controlling the virtual environment. Modern software solutions can detect hands quite accurately and can see and accurately reflect changes in the position of hands, but cannot understand the purpose for which concrete movement of hands in space is performed. In this study, we considered the issue of understanding user movements to create a realistic educational simulator of juggling in mixed reality. We proposed a solution to determine the behavioral cause of the user’s hands to improve the accuracy of the simulator.