Differential co-occurrence analysis: a method to extract ecological modules from clinical microbiome data

The human microbiota plays a pivotal role in health, with widespread alterations implicated in conditions ranging from inflammatory disorders to cancer. While correlation-based network analyses have illuminated ecological interactions within these communities, the host environment uniquely mediates microbial relationships, demanding new methods to capture dynamic, condition-dependent modules of species interactions. Here, we present a statistical framework termed differential co-occurrence analysis, which identifies blocks of taxa whose collective presence is strengthened or weakened under distinct host states. By leveraging recent advances in metagenomics that enable detailed taxonomic profiling and higher-order interaction discovery, our method transcends traditional pairwise correlation constraints. Conceptually akin to associative rule mining, it diverges through the integration of robust statistical modeling, directly extracting interactions that differ significantly between conditions. This approach offers a refined lens to dissect microbiota ecology and could pave the way for new insights into microbiome-associated disease mechanisms. IMPORTANCE: The research on the role of the intestinal microbiota in the onset of cancer and as a modulator of anticancer treatments, including chemotherapeutics and immune checkpoint inhibitors, is helping medicine to identify novel strategies for cancer prevention, for the delivery of more effective treatments, and in reducing treatment side effects and complications. Within this context, it is of crucial importance to approach the analysis of clinical microbiome data with an ecology-oriented perspective and to develop bioinformatics tools able to identify functional interactions in bacterial communities of patients from observational cohort studies. Clinical microbiome datasets are typically high dimensional, comprising numerous taxa measured across relatively few samples. This imbalance increases the risk of statistical overfitting and undermines the robustness of analytical findings. However, recent advances in metagenomic bioinformatics pipelines and reference databases have enabled the comprehensive extraction of genetic information from microbiome samples, facilitating the precise characterization of bacterial species presence and absence. In our manuscript, we describe a statistical computational method that we named differential co-occurrence analysis, which focuses on the analysis of the co-presence of microbiota taxa across samples associated with different host conditions. The proposed method can reveal modules of interacting taxa that are strengthened or weakened when the host condition changes (e.g., when passing from a healthy state to a disease state). The method is general and applicable to a broad range of ecological datasets featuring presence/absence data structures. Furthermore, the method accommodates the analysis of higher-order co-occurrence patterns beyond pairwise co-occurrence, thereby enabling the investigation of higher-order interactions, whose detection and identification are a major challenge in ecological network analysis.