Cross-organ transfer learning for tumor microenvironment classification from colorectal to gastric cancer histopathology

Abstract Transfer learning offers a promising strategy for extending computational pathology models from data-rich to annotation-scarce cancer types, yet the transferability of tumor microenvironment (TME) representations across gastrointestinal organs remains underexplored. Here, we trained Swin Transformer, ConvNeXtV2, and UNI2-h deep learning models on ~ 100,000 annotated colorectal cancer histopathology patches (NCT-CRC dataset) and evaluated their generalization to the HMU-GC gastric cancer dataset (31,096 patches) under zero-shot and few-shot conditions. In the zero-shot setting, evaluation was performed on the full HMU-GC dataset, whereas each few-shot experiment used independently generated training, validation, and test splits. Zero-shot transfer yielded limited performance (macro-F1: 49.15–53.63%), revealing a substantial domain gap between colorectal and gastric histology. Few-shot adaptation with only 5% labeled gastric patches improved macro-F1 to 63.25–68.53%. The best configuration—Swin Transformer with Reinhard stain normalization and 20% labeled target data—achieved 72.26% macro-F1 and 72.22% accuracy, corresponding to an absolute accuracy gain of 18.49% points over the same model’s zero-shot baseline. These findings provide preliminary, single-pair evidence that cross-organ transfer of TME representations between colorectal and gastric histology is feasible but constrained by domain differences. Establishing generalizable cross-organ transfer will require validation across multiple source and target cohorts.