Analysis and mitigation of equipment-induced shortcuts in AI models for laparoscopic cholecystectomy

Machine learning models have seen a lot of success in medical image segmentation domain. However, one of the challenges that they face are confounders or shortcuts: spurious correlations or biases in the training data that affect the resulting models. One example of such confounders for surgical machine learning is the setup of surgical equipment, including tools and lighting. Using the task of identification of safe and dangerous zones of dissection in laparoscopic cholecystectomy images and videos as a use-case, we inspect two equipment-induced biases: the location of surgical tools in the field of view and the direction of lighting, both of which are tightly related to the region of interest. We propose methods for evaluating the severity of these biases based on measurements of model consistency under real or simulated exposure to these shortcuts, and augmentation-based methods for mitigating them. We show that our tool bias mitigations based on pasting tool images in random locations improve the models’ prediction consistency under tool movements by 9 percentage points in the most inconsistent cases, and by 4 percentage points on average. Our lighting bias mitigations based on simulated lighting adjustments help reduce fraction of pixels originally predicted as belonging to the dangerous zone that may flip to safe under light changes from 5% to 1.5%, without compromising segmentation quality.