Designing Multimodal Interactions in Medical Augmented Reality

Surgical procedures demand exceptional spatial and temporal coordination, precision, and advanced motor skills, challenging surgeons' cognitive abilities.Although numerous augmented reality (AR) systems have been developed to assist in the precise localization of target regions through anatomical visualizations, they often add visual complexity and increase the information burden during high-intensity tasks.In these situations of information overload, multisensory feedback can help to disambiguate unisensory representations, resulting in more robust interpretations and enhancing task performance.This work aims to design and evaluate multimodal interactions in AR that enhance user performance and reduce cognitive load during surgical procedures, focusing primarily on audiovisual interactions.Preliminary findings indicate that audiovisual interactions improve accuracy and reduce cognitive load during surgical alignment and localization tasks.Future work might explore cognitive load-adaptive audiovisual augmentations, which could further enhance surgeon performance and user experience.