OrthoSimulator: Empirically trained simulations for ground-truth benchmarking of orthology inference

Orthology inference is foundational for comparative genomics, enabling the reconstruction of phylogenetic relationships and transfer of biological knowledge between species. Methods often infer different orthogroups and orthologous relationships, making robust benchmarking essential for evaluating their accuracy. Existing resources such as the Quest for Orthologs benchmark service and OrthoBench provide valuable benchmarks grounded in real biological data and expert curation. However, the true evolutionary histories in all of these benchmarks are unknown, and therefore the true accuracy of orthology inference methods is unknown. Simulations provide an approach to evaluate accuracy where the complete ground truth is known. However, simulations are only useful for benchmarking if they realistically reflect empirical data. Here, we introduce OrthoSimulator, a Python package for generating realistic, empirically calibrated benchmark datasets with known orthogroups, orthologous relationships, gene trees, and evolutionary events. We demonstrate its utility by benchmarking widely used orthology inference methods across simulations calibrated from multiple methods and empirical datasets. We show that method-to-method variability in ortholog inference accuracy is robust to variation in simulation parameters. We further provide a detailed evaluation of ortholog inference accuracy through independent cross-validation, and show that OrthoFinder is the most accurate of the four methods tested. OrthoSimulator is freely available at https://github.com/lauriebelch/OrthoSimulator.

Authors

Institutions

Publication Details

Journal
bioRxiv (Cold Spring Harbor Laboratory)
Published
2026-11-16
DOI
https://doi.org/10.64898/2026.11.12.766176
Primary Topic
Genomics and Phylogenetic Studies
Type
preprint

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

OrthoSimulator: Empirically trained simulations for ground-truth benchmarking of orthology inference

Steven L. Kelly, Laurence J. Belcher, Jonathan Holmes
2 links
bioRxiv (Cold Spring Harbor Laboratory)
Genomics and Phylogenetic Studies
preprint

OrthoSimulator: Empirically trained simulations for ground-truth benchmarking of orthology inference

Steven L. Kelly, Laurence J. Belcher, Jonathan Holmes
preprint en

Abstract

Orthology inference is foundational for comparative genomics, enabling the reconstruction of phylogenetic relationships and transfer of biological knowledge between species. Methods often infer different orthogroups and orthologous relationships, making robust benchmarking essential for evaluating their accuracy. Existing resources such as the Quest for Orthologs benchmark service and OrthoBench provide valuable benchmarks grounded in real biological data and expert curation. However, the true evolutionary histories in all of these benchmarks are unknown, and therefore the true accuracy of orthology inference methods is unknown. Simulations provide an approach to evaluate accuracy where the complete ground truth is known. However, simulations are only useful for benchmarking if they realistically reflect empirical data. Here, we introduce OrthoSimulator, a Python package for generating realistic, empirically calibrated benchmark datasets with known orthogroups, orthologous relationships, gene trees, and evolutionary events. We demonstrate its utility by benchmarking widely used orthology inference methods across simulations calibrated from multiple methods and empirical datasets. We show that method-to-method variability in ortholog inference accuracy is robust to variation in simulation parameters. We further provide a detailed evaluation of ortholog inference accuracy through independent cross-validation, and show that OrthoFinder is the most accurate of the four methods tested. OrthoSimulator is freely available at https://github.com/lauriebelch/OrthoSimulator.

bioRxiv (Cold Spring Harbor Laboratory)
University of Oxford (GB)
Wellcome Trust
Genomics and Phylogenetic Studies
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon