Stability-based partitions for ensemble clustering

Unsupervised clustering is challenging because there is no gold standard, making cluster quality assessment difficult. Clustering solutions depend on several choices the investigator makes, including the algorithm, hyperparameters, measures of dissimilarity, and the number of clusters. Stability has become a valuable tool for characterizing the quality of a clustering solution and prioritizing clusters. This work presents Stability-based Partitions for Ensemble Clustering (SPEC) to combine partitions from different clustering solutions to produce aggregate clustering. SPEC utilizes a graph-based representation of partitions with edge weights calculated from normalized item-level subsampled stability within a partition. Community detection with weighted bipartite graphs identifies the aggregate clusters within the graph. SPEC can aggregate solutions within a clustering algorithm with different hyperparameters across different clustering algorithms and integrate multi-modal data. SPEC applications to simulation and real data reveal that the stability-weighted graph is robust to noise and spurious solutions and can emphasize different geometries in the feature space. Applications to a METABRIC (Molecular Taxonomy of Breast Cancer International Consortium) dataset show that SPEC combined clinic data, gene expression data, and genetic mutation data to identify subpopulations of patients with differential survival profiles.