Design and Clinical Evaluation of ARAS: An Augmented Reality Assistance System for Open Pancreatic Surgery

The integration of Augmented Reality (AR) technology into surgical procedures offers significant potential to enhance clinical outcomes. While there are plenty of lab-proven prototypes, systems employed in actual clinical settings require specialized design and rigorous clinical evaluation of these AR-based solutions to meet the high demands of complex medical fields. Our research exposes these complex requirements emerging from clinical environments, such as operation theaters. To address the challenges, we introduce ARAS, an operational AR assistance system for live open pancreatic surgery. Employing a user-centric design methodology, we designed and refined ARAS through several iterations, ensuring its practical applicability and effectiveness in a real-world surgical setting during clinical trials. ARAS enables in situ and precise visualization of the patient’s 3D reconstructed vascular system and tumor during the surgical procedure. We evaluated ARAS through clinical trials (N=7) involving patients diagnosed with pancreatic cancer. Our interviews with the surgeons underscored the utility of ARAS for open pancreatic surgery, especially in critical and highly time-pressured phases of the surgery, as it proved to be exceptionally beneficial in aiding surgeons during in their decision-making process. In a post-surgery evaluation, the surgeons also certified the precise visualization accuracy of ARAS during those critical phases. Our findings showcased the heightened requirements for AR-based solutions in operational clinical use and proved that ARAS met the challenges emerging during live surgeries. Consequently, surgical AR assistance systems do have a transformative potential to revolutionize traditional practices, but their applicability is subject to high design constraints during critical medical procedures.