Scalable Incremental Clustering for Tandem Mass Spectra in Untargeted Metabolomics

Tandem mass spectrometry (MS/MS) has become the analytical backbone of large-scale untargeted metabolomics, routinely generating millions of spectra per study. However, existing clustering methods struggle to process this scale due to computational and memory bottlenecks, limiting the utility of clustering in downstream analysis. This bottleneck is especially acute in long-term studies and public repositories, where new data are continuously added over time. Here we present a scalable clustering framework for MS/MS metabolomics data. Our method incrementally incorporates new spectra batches while preserving clustering performance through a novel spectrum pooling strategy, which propagates local density structure across batches. Using both database-search-based evaluation on proteomics data sets and the MS1-retention time (MS-RT) method on metabolomics data sets, we show that incremental clustering achieves comparable performance to the state-of-the-art clustering methods in terms of cluster purity and completeness. Critically, our approach scales up to clustering tasks consisting of 368 million spectra clustering task and millions of clusters, completing in under 10,000 CPU hours, while traditional methods could not scale to this data volume and failed to complete due to excessive memory or time requirements. Our method offers a practical solution for large-scale, continuously growing MS/MS studies and is well suited for integration into public metabolomics platforms such as GNPS2.