Eye Movement-Based Human–Swarm Interaction for Coverage Control of Mobile Robots With Constraints

Inspired by search and rescue tasks in hazardous environments with mixed human and robots teams, we develop a human–swarm interaction scheme for coverage control of mobile robots in a non-convex and closed 2-D area. Motivated by releasing the manual operations of a human, we develop two eye movement-based interaction modes, namely eye gazing mode and eye tracking mode, which use Gaussian functions to approximate signal density functions with respect to a human operator's focus on the specific area of interest. These density functions are then utilized to generate controllers for the robots to achieve the optimal coverage objective with human influence. The coverage controller is designed considering several practical constraints and requirements, including 1) the non-convex area; 2) the limited sensing radius; 3) the time-varying nature of the density function; and 4) the requirement for multiple sensors to monitor an area. To solve these challenges, we design a gradient-based controller by utilizing the visibility-limited higher order Voronoi partition technique, the geodesic distance metric and a well-designed performance criteria function. The implementations and experiments using a Microsoft Hololens 2 Augmented Reality headset and multiple turtlebot robots are presented.