Human vision-assistive control based on motion illusion for drone-swarm cooperative control in a VR environment

This paper proposes a field-of-view (FoV) assistive control technique called redirected walking (RDW) that smoothly links human motion in the real world and virtual reality (VR) space by maneuvering through motion illusion. In particular, to improve the cooperative consensus control between the human operator equipped with the head mount display in the real world and the drone swarm in VR space, one technical issue is to apply the RDW technique to a dynamical formulation implementable in real-time. To overcome the issue, we propose a dynamic RDW technique based on sigmoid gain mapping. To evaluate the quantitative performance of the proposed control method, we have created a new experimental environment with Unreal Engine 4, AirSim, Meta Quest3, and Python. The effectiveness of the proposed method is finally demonstrated through the developed experimental environment. As a result, we see that the proposed method can improve the drone swarm’s tracking and maneuvering performance and suppress discrepancies between somatosensory perception and the human operator’s FoV.