Nested Gaussian mixture model: clustering data from generalized mixture distributions

Hierarchical clustering and the Gaussian mixture model are two important means of cluster analysis. As a generative model, a well-trained Gaussian mixture model can naturally characterize the distribution of data consisting of Gaussian clusters and thus gain the ability to handle incremental data. However, the validity of Gaussian mixture model relies on the consistency between data distribution and model assumption. That is, it requires particular knowledge about the prior distribution of a dataset for the user to decide whether Gaussian mixture model is competent since GMM fails to model data from non-Gaussian mixture distribution. By contrast, agglomerative nesting is a nonparametric, hierarchical clustering algorithm that does not require prior knowledge. In practice, agglomerative nesting tends to fluctuate drastically in performance with different similarity metrics and requires a lot of additional computing resources as the sample size grows. This paper proposes an arbitrary mixture model for characterizing data from unknown mixture distribution. In the absence of information about the prior distributions of clusters, we leverage a nested mixture model to approximate the unknown mixture distribution. This is achieved by decomposing the unknown distributed clusters and remodeling them with an equal number of Gaussian mixtures. Particularly, we implement the two successive procedures of the method achieved by using the EM algorithm with incomplete covariance and SingleLink agglomeration on a compressed similarity matrix. Experimental results demonstrate that our method shows clear advantages over the Gaussian mixture model and agglomerative nesting in both runtime and performance, improving clustering accuracy by 12–25% across standard evaluation indices and reducing the cluster number error to 12, compared with 70 and 151 for competing Gaussian mixture-based methods.