Machine Learning-Driven Prediction of Gleason Score 7 Prostate Cancer Patterns Using Multi-Omics Data

This study aims to develop a novel advanced Prostate Cancer (PCa) prediction system that utilizes vital omics (DNA Methylation, Gene Expression, and Copy Number Alteration) combined with gene-specific mutation features to stratify 3 + 4 and 4 + 3 Gleason Score samples. Although the 3 + 4 and 4 + 3 samples are variants of grade-7 PCa, the latter is more severe. Histopathological and clinical similarities between these classes often lead oncologists to misdiagnose them. The utilized dataset in this study contains the aforementioned omics data for the most mutating genes (including SPOP, FOXA1, DPYSL2) with Gleason Score as the target, combined with the mutation profiles for 11,690 genes. This study used a Conditional Tabular Generative Adversarial Network (CTGAN) for handling class imbalance resulting in 145 samples in each class. This dataset was subjected to standard scaling and binary classification using GridSearchCV tuned models such as XGBoost. Mutation features from highly mutating genes were then combined, followed by dimensionality reduction using PCA and a shallow Autoencoder. Binary classification from here on was done with custom deep ANN classifiers in addition to the hyperparameter tuned ML models. An alternate approach to achieve more accuracy was followed where a deep Autoencoder was used to obtain 13 latent features from the dataset. Recursive Feature Elimination (RFE) in combination with a Random Forest estimator, were used to obtain the highest-ranking latent features. Isolation Forest and One-Class Support Vector Machine (O-SVM) were then used to effectively stratify the samples. In addition, Kullback Liebler Divergence (KLD), a probability and information loss-based approach was used to study whether all the 3 + 4 and 4 + 3 samples varied significantly. XGBoost achieved the best accuracy (66–71%), outperforming ANN-4 (66%), Random Forest (62%), and a Max-Vote classifier (61%). The system effectively integrates multi-omics data and gene mutations, achieving robust classification despite data limitations.