S682 AI-Powered Multimodal Prognostic Model for Colorectal Cancer Risk Stratification Using Integrated Image and Pathology Data

Introduction: Colorectal cancer accounted for more than 1.9 million new diagnoses and 935 000 deaths worldwide in 2020, ranking it as the third most common malignancy and the second leading cause of cancer-related mortality. Although deep learning has achieved 96% accuracy in real-time polyp detection, most models rely exclusively on visual features and omit patient metadata, limiting their diagnostic scope and translational potential. Methods: We aligned 2-fps video frames and high-resolution still images from the Endoscopic Image and Video Dataset for Recognition of Colorectal Polyps Morphology and Pathology-v5 dataset—comprising 430 anonymized images and videos from 217 patients—with structured pathology metadata via unique patient identifiers. We resized all images to 224 × 224 pixels and normalized pixel intensities before extracting 768-dimensional feature vectors using a pre trained Vision Transformer. Continuous variables (e.g., polyp size) underwent scaling, and categorical fields (e.g., dysplasia grade, histological subtype) received one-hot encoding. We concatenated these embeddings with the processed metadata to form a unified feature set. To prevent data leakage, we partitioned patients into training (60%), validation (20%), and test (20%) cohorts. Finally, we trained a Random Forest classifier for binary malignancy prediction and devised a probabilistic scheme to stratify cases into low, moderate, or high risk based on predicted malignant probabilities. Results: On the independent test cohort, the malignancy classifier reached 99.58% accuracy and achieved perfect discrimination (area under the receiver operating characteristic curve [AUC-ROC] = 1.00). It demonstrated 96% sensitivity for malignant lesions and an F1-score of 0.98, reflecting highly reliable detection of high-risk polyps. The resultant probability scores enabled clear assignment into clinically relevant risk tiers, which aligned closely with histopathological findings and supported rapid, real-time triage during colonoscopy. Conclusion: The proposed multimodal framework delivered 99.6% accuracy for malignancy prediction with perfect discrimination (AUC-ROC = 1.00) on an independent patient-held cohort, facilitating clear stratification into low, moderate, and high-risk tiers. By integrating Vision Transformer–derived image embeddings with structured clinical data, this approach promises standardized, real-time polyp assessment and supports more consistent decision-making in colorectal cancer screening.