3DGS-based Tracking of Articulated Instruments for Instruments-Tissue Interaction Estimation in Robot-assisted Minimally Invasive Surgery

3D information of surgical scene potentially enhances the accuracy of surgical navigation and facilitate the automation of robot-assisted surgery. However, current research seldom considers the relative 3D position between instruments and tissue. In this paper, the instruments-tissue interaction estimation method is proposed to obtain the relative distance between instruments and tissue. The 3D Gaussian representation of instruments is established preoperatively, making the pose of instruments the only parameter to be learned and updated. Based on this, the instrument pose estimation method is proposed, which applies 3DGS (3D Gaussian Splatting) to the instrument Gaussian model to obtain rendered color images, projection masks, and depth maps. These are aligned with the input images, segmented masks and depth from stereo depth estimation to construct the target loss. By backpropagating through the target loss, the optimal instrument pose is obtained. The L1 loss and smooth loss based on TPS (the Thin Plate Spline) are used to complete the depth of tissue occluded by instruments. The instruments-tissue interaction is estimated by the nearest distance between instrument and tissue. The experimental results indicate an average error of 0.94 mm and an accuracy of 90.40% with an error less than 2mm in estimating relative 3D distance between instruments and tissue.