Compact and informative representation learning for scRNA-seq data clustering with masked information bottleneck

BACKGROUND: Single-cell RNA sequencing (scRNA-seq) provides unprecedented opportunities to characterize cellular heterogeneity. However, the high sparsity, noise, and redundancy inherent in gene expression data often obscure biologically meaningful signals and hinder accurate cell clustering. Although highly variable genes are commonly used to reduce dimensionality, they may still contain redundant or noisy information that degrades clustering performance. RESULTS: Here, we propose scMIB, a masked information bottleneck framework for robust representation learning in scRNA-seq data. The method introduces a masking-based denoising strategy that perturbs gene expression patterns and trains the model to recover informative structures while suppressing noise. By integrating an information bottleneck objective, scMIB further compresses redundant signals and preserves the most relevant information for clustering. In addition, a mask consistency learning mechanism is employed to align real and predicted masks, encouraging the model to capture stable gene-level patterns. Extensive experiments on multiple public scRNA-seq datasets demonstrate that scMIB consistently improves clustering accuracy and robustness compared with existing methods, while effectively mitigating the influence of noise and sparsity. CONCLUSIONS: Our results show that combining masking-based perturbation with information bottleneck learning provides an effective strategy for extracting informative representations from noisy single-cell transcriptomic data. The proposed framework offers a robust solution for scRNA-seq clustering and may facilitate more reliable identification of cellular heterogeneity in complex biological systems.