Image-guided surgery and examinations using ultrasound

Many of today’s cancer surgeries are carried out with robot assistance. Using real-time intra-operative ultrasound, we can overlay pre-operative imaging into the surgeon’s console, enabling visualization of sub-surface anatomy and cancer at the same time with the standard laparoscopic camera view. We will discuss aspects of system design, visualization and registration methods that enable such visualization, and present our results. We will also present tissue and instrument tracking approaches that can be used in future augmented reality systems. For remote and underserved communities, we developed a teleultrasound approach that relies upon using a novice – the patient, a family member or friend – as a robot to carry out the examination. The novice wears a mixed reality headset and follows a rendered virtual ultrasound transducer with the actual transducer. The virtual transducer is controlled by an expert, who sees the remote ultrasound images and feels the transducer forces. This tightly-coupled expert-novice approach has advantages relative to both conventional and robotic teleultrasound. We discuss our system implementation and results.