Mixed Reality Interface for Managing Heterogeneous Robot Teams in Complex Environments

Mixed Reality (MR) interfaces have been widely used for controlling single mobile robots, but their application for managing teams of mobile robots remains an under-explored area. This extended abstract presents preliminary work on HORUS: Holistic Operational Reality for Unified Systems, a Mixed Reality interface designed to manage diverse robot teams, including wheeled, legged, and aerial robots. HORUS enables operators to monitor robot status, view real-time sensor data, and assign tasks to individual robots, subsets of the team, or the entire team via a Mini Map (Ground Station). The system supports teleoperation with bespoke control interfaces tailored to different robot types, ensuring optimal performance for various platforms. While the wheeled robot multi-robot teleoperation functionality has been fully implemented, work is ongoing to integrate multi-task assignments for wheeled robots and then integrate legged robots and drones into the system.