Computer-Vision Approach to Triaging Patient-Submitted Photos of Intestinal Ostomies

BACKGROUND: Surgeons and ostomy nurses receive a high volume of stoma photos from patients. OBJECTIVE: This study aimed to develop and validate an automated and scalable artificial intelligence pipeline for identification and triage of patient-submitted intestinal ostomy photos. DESIGN: Retrospective derivation and validation study with stakeholder engagement to guide model design. Clinical teams reviewed photos in duplicate and categorized them into: (a) obstructed view, (b) healthy stoma, (c) suitable for conservative management, or (d) requiring in-person review. Pre-trained neural networks including MobileNetV4, ResNet50, ViT and CLIP-ViT were fine-tuned with 5-fold cross-validation. SETTINGS: Nine Mayo Clinic hospitals (2019-2022). PATIENTS: Adult patients (≥18 years) undergoing surgery who submitted images within 30 days after surgery. MAIN OUTCOME MEASURES: Model performance was evaluated with area under the receiver operator curve, precision, recall, and F1-scores. RESULTS: In this study, 538 photos of abdominal ostomies were sent in by 191 patients (median age 52 years, IQR 40-63; 60.4% female). Expert consensus triaged 236 (43.9%) photos as having an obstructed view, 30 (5.6%) as healthy, 177 (32.9%) for conservative management, and 95 (17.7%) for in-person review. All models achieved area under the receiver operator curve >0.97 for identifying stomas. The CLIP-ViT model performed best at triage (macro- area under the receiver operator curve 0.94 ± 0.06; F1 0.77 ± 0.10). End-to-end detection and triage using CLIP-ViT achieved area under the receiver operator curve of 0.99 ± 0.01, precision of 0.87 ± 0.08, recall of 0.93 ± 0.04, and F1 of 0.89 ± 0.07. Attention maps showed that models focused on stomas to determine classification. LIMITATIONS: Low sample size and lack of prospective validation. CONCLUSIONS: A vision language model pipeline accurately detected and triaged patient-submitted ostomy photos. Prospective evaluation is now needed to support integration into multidisciplinary digital workflows. See Video Abstract.