DEEP MULTI-MODAL REPRESENTATIONAL PHENOTYPING OF GLIOMA

Abstract AIMS Complex heterogeneity is a major obstacle to clinical and scientific progress in neuro-oncology, resolvable only with highly expressive models powered by large-scale data and high-performance compute. We sought to de- velop a comprehensive multi-modal modelling framework with sufficient expressivity to capture heterogeneity distributed across the multiple characteristics that distinguish individual glioma. METHODS Evaluating a multi-site cohort of 4908 glioma patients, we employed multi-modal spatial autoencoders to derive succinct, spatially organised latent representations of anatomically organised patterns of heterogeneity. We quantified the prognostic and diagnostic utility of these representations within XGBoost discriminative models and compared their performance against conventional brain tumour featurisation Methods radiomics and VASARI. RESULTS The best prognostic models combined our deep autoencoder representations, VASARI scores, and radiomics, with histology (test set c-index=0.74). Prognostic models with the spatial autoencoder representations alone achieved a c-index of 0.71, surpassing both VASARI (c-index=0.69) and radiomics (c-index=0.69). The best diag- nostic model combined our deep autoencoder representations, VASARI scores, and radiomics, successfully dis- tinguishing glioblastoma from oligodendroglioma and astrocytoma with a test set balanced accuracy of 92.6%. The deep autoencoder alone could predict diagnosis with a balanced accuracy of 82.1%, while VASARI alone achieved 81.7% and radiomics alone reached 80.5%. CONCLUSION Deep representations drawn from multimodal imaging data successfully capture heterogeneity opaque to con- ventional glioma characterisation and carry potentially useful predictive and diagnostic signals. These models may facilitate better patient care personalisation with more robust image-guided diagnostics and long-term outcome monitoring. Future research will harness these models to uncover new mechanistic insights.