Towards Interactive 3D Surgical Scene Reconstruction: An Incremental Training and Monitoring Framework

Dynamic 3D scene reconstruction is essential for minimally invasive surgeries, providing enhanced visualization to assist surgeons during complex procedures. However, existing methods lack real-time interaction, limiting surgeons to post-surgical inspection of the reconstructed 3D scenes. Additionally, these approaches often struggle with varying endoscopic views due to camera movement or surgical manipulation, both of which are common during surgery. Addressing these challenges is difficult because an accurate 3D scene reconstruction model is often computationally intense and demands extensive training. To bridge the gaps, we present a prototype framework that enables near-real-time 3D scene reconstruction with simultaneous expert interaction. Specifically, our approach integrates an interactive front-end interface with incremental back-end model training, managed by a periodical monitoring unit, allowing surgeons to interact with and monitor the reconstruction process throughout the procedure. We hope this work inspires further development of interactive systems for surgical support. Our future efforts will focus on extending this prototype into real-world applications.