scDETECT: a novel statistical model accounting for cell type correlation in single-cell RNA-seq differential expression analysis

Differential expression (DE) is one of the most important analyses in single-cell RNA-seq (scRNA-seq). Due to similarity of cell types, the DE states often have strong correlation among different cell types. Existing methods perform DE analysis for each cell type separately and ignore such correlation, leading to low accuracy, and statistical power. We develop single cell Differential Expression TEst with Cell Type correlation (scDETECT), a novel statistical method, for scRNA-seq DE analysis accounting for the cell type correlations. scDETECT implements a Bayesian hierarchical model to incorporate the cell type correlations into the modeling of the gene expression, and then the DE genes are called based on the derived posterior probabilities. Simulation and real data studies show that scDETECT significantly improves the accuracy and statistical power compared with existing methods.