OrthoSimulator: Empirically trained simulations for ground-truth benchmarking of orthology inference
Orthology inference is foundational for comparative genomics, enabling the reconstruction of phylogenetic relationships and transfer of biological knowledge between species. Methods often infer different orthogroups and orthologous relationships, making robust benchmarking essential for evaluating their accuracy. Existing resources such as the Quest for Orthologs benchmark service and OrthoBench provide valuable benchmarks grounded in real biological data and expert curation. However, the true evolutionary histories in all of these benchmarks are unknown, and therefore the true accuracy of orthology inference methods is unknown. Simulations provide an approach to evaluate accuracy where the complete ground truth is known. However, simulations are only useful for benchmarking if they realistically reflect empirical data. Here, we introduce OrthoSimulator, a Python package for generating realistic, empirically calibrated benchmark datasets with known orthogroups, orthologous relationships, gene trees, and evolutionary events. We demonstrate its utility by benchmarking widely used orthology inference methods across simulations calibrated from multiple methods and empirical datasets. We show that method-to-method variability in ortholog inference accuracy is robust to variation in simulation parameters. We further provide a detailed evaluation of ortholog inference accuracy through independent cross-validation, and show that OrthoFinder is the most accurate of the four methods tested. OrthoSimulator is freely available at https://github.com/lauriebelch/OrthoSimulator.
Authors
- Steven L. Kelly (ORCID: https://orcid.org/0000-0001-8583-5362)
- Laurence J. Belcher (ORCID: https://orcid.org/0000-0003-2392-6503)
- Jonathan Holmes (ORCID: https://orcid.org/0009-0001-0440-8404)
Institutions
- University of Oxford (GB)
Publication Details
- Journal
- bioRxiv (Cold Spring Harbor Laboratory)
- Published
- 2026-11-16
- DOI
- https://doi.org/10.64898/2026.11.12.766176
- Primary Topic
- Genomics and Phylogenetic Studies
- Type
- preprint
Funders
- Wellcome Trust