scDVAE:Single-Cell Data Clustering Based on Variational Autoencoder With Disentangled Latent Representations

Single-cell RNA sequencing (scRNA-seq) technology enables the analysis of gene expression in individual cells, allowing for a deeper exploration of heterogeneity in organisms and complex diseases. Cell clustering is a crucial step in single-cell analysis, enabling the identification of cellular heterogeneity. However, the high dimensionality, sparsity, and dropout events in single-cell data have brought enormous challenges to clustering analysis. Building on the proven success of deep generative models in learning meaningful representations from low-dimensional latent spaces, we introduce scDVAE, a novel deep generative approach that leverages a variational autoencoder with disentangled latent representations for single-cell clustering. First, each latent representation generated by the encoder is disentangled into clustering features and generative features. In this way, the clustering features can enhance the performance of the clustering task without interference from the generative task. Second, we employ a Student's t-mixture model as the prior distribution for the clustering features to enhance the robustness of our method against dropout events. In addition, we introduce a hybrid data augmentation strategy to generate augmented scRNA-seq data, which enhances dataset diversity while also helping to reduce noise. Our experimental studies on 10 real-world datasets demonstrate that scDVAE significantly improves clustering performance compared to state-of-the-art methods.