HySim: towards development of a hybrid simulation framework with improved visual and tactile realism for minimally invasive surgeries

Abstract Simulation-based training is essential for developing minimally invasive surgical skills. While box trainers provide tactile feedback for practicing techniques like suturing and knot-tying, they lack visual realism. Virtual Reality (VR) simulators provide a visually realistic environment but often lack tactile feedback. This work integrates the strengths of both box trainers and VR simulators in a simulation framework. The proposed simulation framework, HySim, combines virtual and physical tissues into a unified operative view, thereby improving visual and tactile realism. HySim was assessed on: (a) simulation of surgical scenarios, (b) feedback from 10 participants specialized in robotic urology surgery on visual and tactile realism, and usefulness, and (c) the alignment of virtual and physical tissues during scope movements. HySim was successfully deployed for both laparoscopic (bowel anastomosis) and robotic (urethral dissection) scenarios. Compared to VR simulator, surgeons rated HySim higher for instrument motion and handling ( p = 0.042), tool-tissue interaction ( p = 0.022), tissue movement and behavior ( p = 0.004), and usefulness ( p = 0.042). It was also rated higher compared to box trainer for the operative view ( p = 0.042). However, all three simulation environments received similar scores for the likelihood of adoption. For a scope tilt/pan movement with speed less than 10 degrees per second, the physical and virtual tissues did not have noticeable misalignment. HySim can enhance the training environment by improving both visual realism and tactile feedback. Further user studies are required to assess HySim for simulating generic and patient-specific preoperative training scenarios.