Deformation Description and Tracking in Digestive Endoscopy

Soft tissue surface tracking and 3D reconstruction are crucial for augmented/mixed reality navigation systems. However, existing methods struggle to accurately describe, match, and track deformations in digestive endoscopic images. This paper proposes a novel method for detecting, semantically describing, and tracking soft tissue surface deformations. First, an adaptive contour detection algorithm identifies deformation regions in gradient space by locating the inflection point of the density distribution function. Next, an MLP-based encoder-decoder network achieves efficient semantic coding of deformation contours, enabling high-level semantic compression from unlabeled datasets. A directed graph structure is then constructed to model deformation areas, calculating node direction and displacement to track soft tissue changes. Compared to state-of-the-art methods, our approach significantly improves deformation description and performs best across various endoscopic images. Experiments demonstrate its accuracy in detecting, describing, and tracking soft tissue deformations, showing potential for augmented/mixed reality applications.