A Collaborative Mixed Reality Platform for the Planning of Vascular Surgery

In this paper, we develop a mixed reality (MR) platform for the collaborative visualization and manipulation of 3D images of the human body and the control of numerical simulations for the purpose of surgical planning for certain vascular diseases. The MR platform allows multiple users to view and make changes of virtual objects in a server/client fashion in real time. The input of the system is the images of a human body obtained from MRI scans and segmentations of some of the organs are obtained as a pre-processing step. The interaction of the physical hands and the virtual objects are carried out through several pairs of holographic devices equipped with hand gesture recognition. For each planned surgical configuration, the corresponding hemodynamics is analyzed via a connected supercomputer whose output is transferred back to the controlling holographic devices. As an example, we consider the surgical planning of the type B aortic dissection. This is a rather complex procedure and the proposed mixed reality platform is used to visually study several possible placement of a stent. In addition to visualization, the platform also provides near real time computation of the blood flows corresponding to each of the planned surgical scenarios, and the computation is controlled by hand gestures from the holographic devices.