A comparison of audible, visual, and multi-modal communication for multi-robot supervision and situational awareness

Multi-robot supervision becomes increasingly cognitively demanding as the ratio of robots to human supervisors rises, potentially leading to situational awareness (SA) losses and robot system failures. Nonverbal cues have been employed to direct supervisor attention and prevent awareness loss in diverse human-computer interaction (HCI) settings. This paper compares the effects of uni-modal and multi-modal audiovisual nonverbal cues on supervisor SA in a multi-robot supervision task. In a simulation-based navigation scenario, 50 participants monitored a multi-robot mission and responded to supervision requests from the robots. We evaluated supervisor SA using response speed and the situational awareness global assessment technique. Results demonstrate that supervisor awareness hinges on the communication method employed by the robots, with greater significance observed at higher awareness levels and when the robot-to-human ratio is higher. Findings also indicate the utility of sonification mapping in human-multirobot interactions and the benefits of multi-modal cues for sustaining awareness during multi-robot supervision.