A living benchmark to advance AI in proteomics de novo peptide sequencing

Deep learning has enabled a new generation of de novo peptide sequencing tools for identifying peptides directly from MS/MS spectra, with applications across diverse areas of biology, biotechnology, and medicine. Yet, their evaluation remains inconsistent due to the absence of standardized datasets and metrics, limiting users' ability to select the right tool and slowing methodological progress. To address this, we present the first community-driven, living benchmark for de novo sequencing, encompassing 29 diverse proteomics datasets with rigorously validated ground truth annotations. The benchmark currently evaluates 18 tools, each optimally configured by its developers, across amino acid- and peptide-level accuracy, spectral similarity, and reference proteome matching. Results reveal both the promise of modern deep learning models—particularly transformer-based architectures—and persistent challenges, such as non-tryptic proteases, post-translational modifications, and cross-instrument generalization. By providing an open, continually updated evaluation platform, this benchmark delivers an essential foundation for developing, comparing, and deploying AI approaches in proteomics.