Clustering and classification of ovarian cancer samples using machine learning

Ovarian cancer is one of the most lethal gynecological malignancies, largely due to the challenges posed by late diagnosis and metastasis, which complicates treatment and worsens patient outcomes. To better understand the underlying biology and identify potential prognostic markers, we utilized high-quality RNA-Seq transcriptomic data from the TCGA-OV cohort, which includes 390 ovarian cancer samples. Our primary objective was to identify distinct patient clusters within this dataset, with the goal of uncovering patterns that could enhance our understanding of the disease and guide targeted therapeutic approaches. Hierarchical clustering identified three distinct patient groups: A, B, and C. Groups A and B exhibited contrasting gene expression profiles, with Group A enriched in genes related to metastasis and exosome biogenesis, indicative of a more aggressive cancer phenotype. Conversely, Group B was enriched in cilium-related genes, which are associated with less aggressive cancer types such as the differentiated and proliferative subtypes. Group C displayed an "intermediate" gene expression profile, with characteristics that partially overlapped with either Group A or B, as seen in the hierarchical clustering and PCA analyses. This suggested that Group C might not represent a distinct biological entity, but rather a transitional phase between the more clearly defined Groups A and B. To enhance the statistical power and biological significance of our analyses, we reclassified patients in Group C into Groups A and B using a Machine Learning strategy based on Random Forest and k-NN algorithms. This reclassification was not only biologically justified, given the overlapping gene expression profiles, but it also enhanced the statistical significance of downstream analyses by increasing the sample sizes within Groups A and B. This approach ensured that the differences between Groups A and B were more statistically robust and reflective of true biological differences, rather than artifacts of smaller sample sizes. Survival analysis confirmed that patients in Group A had significantly worse outcomes than those in Group B (p=0.016). This was further supported by functional enrichment analyses, which revealed distinct transcriptional programs related to metastasis and exosome biogenesis in Group A, and cilium function in Group B. These findings underscore the importance of the identified gene expression patterns in determining patient prognosis and highlight the potential for these patterns to serve as biomarkers for targeted therapeutic strategies. In conclusion, our study not only identified distinct ovarian cancer patient clusters with varying prognoses but also demonstrated an effective methodology for reclassifying transitional patients to enhance the robustness of comparative analyses.