Development and validation of a multimodal AI-agent system for prognosis analysis of bladder urothelial carcinoma

Precise survival risk stratification for bladder urothelial carcinoma (BUC) remains a clinical challenge. We developed and validated a multimodal AI agent that integrates textual, radiographic, and pathological data from 1185 patients across four medical centers to predict survival risk. The agent employs LLMs to standardize pathology reports, interactive deep learning networks for precise CT image segmentation, and extracts features from CT scans and whole slide images using CTVisionNet and MacroVisionNet. The multimodal fusion framework, MATCH-Net, integrates these features with microscopic pathology information and clinical text embeddings using a multi-head attention mechanism to generate a comprehensive prognostic score. In multi-center validation, MATCH-Net demonstrated robust performance (C-index ranging from 0.836 to 0.874) and effectively stratified patients into high- and low-risk groups, identifying potential candidates responsive to adjuvant chemotherapy. Furthermore, the framework enabled the quantification of novel, interpretable prognostic biomarkers and provides a reliable and clinically applicable solution for personalized BUC prognosis.