Anywhere Projected AR for Robot Communication: A Mid-Air Fog Screen-Robot System

Augmented reality (AR) allows visualizations to be situated where they are relevant, e.g., in a robot's operating environment or task space. Yet, headset-based AR suffers a scalability issue because every viewer must wear a headset. Projector-based spatial AR solves this problem by projecting augmentations onto the scene, e.g., recognized objects or navigation paths, viewable to crowds. However, this solution mostly requires vertical flat surfaces that may not exist in large open areas like auditoriums, warehouses, construction sites, or search and rescue scenes. Moreover, when humans are not co-located with the robot or situated at a distance, the projection may not be legible to humans. Thus, there is a need to create a projectable, viewable surface for humans in such scenarios. In this HRI systems paper, we introduce a fog screen-robot system that integrates a mid-air fog screen device into a robot to create such a projectable surface and presents two evaluations in a construction site and a search and rescue scenario for high-stakes communication needs. Specifically, we implemented an existing fog screen device, which can only project one-third of a meter (33cm). We improved it to achieve a fog screen length of half a meter (53cm). In the noisy construction site scenario, the robot inspected the site and projected icons for missing wall sockets and plumbing fixtures. In the unstructured search and rescue scenario, the robot was able to project a person icon for a first responder to save life. All 3D models and code are available at https://github.com/TheRARELab/fog-screen-robot-system. Videos are available at https://osf.io/b4efu/.