Computer vision augmented indocyanine green near-infrared fluorescence (ICG-NIRF) signal-to-background ratio (SBR) for identification of extrahepatic biliary structures in laparoscopic cholecystectomy

Abstract Introduction Indocyanine green near-infrared fluorescence (ICG-NIRF) improves visualization of extrahepatic biliary ducts (EHBD) compared to white light imaging (WLI) in laparoscopic cholecystectomy (LC). However, the optimal EHBD signal is limited by background hepatic fluorescence disturbance, complicating interpretation of critical structures despite ICG pharmacologic approaches. We describe the feasibility of multi-modal computer vision (CV) models to improve the discrimination of the EHBD ICG signal to liver background ratio (SBR), facilitating intra-operative visualization of critical structures. Methods A multi-institutional LC database was curated ( n = 928). A retrospective analysis of ICG-NIRF LC cases ( n = 156) was performed to identify images of hepatobiliary anatomy in both WLI and ICG-NIRF. The multi-modal images were annotated for EHBD, liver, and instruments. A YOLOv11-based model selectively attenuated background liver ICG-NIRF, to improved EHBD visualization. A novel WLI-ICG fusion image generated after SAM3 segmentation displayed ICG-NIRF enhanced EHBD superimposed on WLI liver parenchyma, improving critical structure identification while eliminating background noise. The visual impact of the models was quantified via intensity of EHBD (signal) to liver (background) ratio. Results The baseline EHBD to liver SBR in ICG-NIRF was 1.21 ± 0.69. Attenuating background liver fluorescence alone improved SBR (4.46 ± 3.06) ( p < 0.0001). The WLI-ICG fusion model displaying ICG enhanced EHBD on native liver parenchyma improved SBR (3.07 ± 1.09) ( p < 0.0001) while improving usability. The multi-modal detection models demonstrated superior performance versus previous models trained on WLI alone, precision: 0.911, recall: 0.866, F1 score: 0.887, mAP: 0.842. Conclusion Our study demonstrates the methodologic feasibility of CV models to selectively enhance the EHBD to liver SBR via generation of a novel, practical WLI-ICG fusion image. Multi-modal CV identification and augmentation of EHBD provides higher quality, more objective assessment of critical structures and a potentially more universal solution to the utility and usability of ICG-NIRF compared to previous pharmacologic approaches.