Advancing Situation Awareness in Safety-Critical Systems: The Role of Multi-Modal Wearable Augmented Reality

Augmented Reality (AR) is emerging as a transformative tool for enhancing multi-modal interaction and situational awareness in safety-critical environments. This dissertation investigates the impact of wearable AR interfaces on maritime navigation, examining their effects on operator performance, cognitive workload, and operator communication. This research evaluates AR’s role in supporting multi-modal information processing and decision-making in high-risk operational contexts. This work is structured around three studies. The first study quantifies how wearable AR displays influence navigational accuracy, cognitive load, and situation awareness, revealing both efficiency gains and cognitive trade-offs. The second study explores multi-modal AR’s impact on operator mobility and communication, highlighting its ability to streamline real-time decision-making while requiring new communication strategies. The third study takes a qualitative approach, capturing operator perceptions of AR usability, trust development, and workflow integration. Findings indicate that while AR can enhance situational awareness and mobility, its adoption depends on interface adaptability, ergonomic design, and user training. This dissertation contributes to the broader discourse on multi-modal interaction in safety-critical systems, demonstrating how wearable AR shapes operator cognition, information flow, and collaborative decision-making. Beyond maritime navigation, these insights extend to aviation, emergency response, and industrial safety. Additionally, reflections on Google Glass and the GlassNav AR system provide design recommendations for future multi-modal AR interfaces. The research also explores the trajectory of AR in an AI-enabled world, emphasizing the need for adaptive, AI-driven AR solutions that dynamically tailor information presentation to operator workload. By identifying key challenges and opportunities in AR integration, this dissertation advances the development of human-centered, multi-modal AR technologies for safety-critical operations.