QRAFT: Scaling coalescent-based species tree inference to one million taxa

Species tree inference from multi-locus data under the multispecies coalescent has not kept pace with phylogenomic datasets that now span hundreds of thousands of species. We present QRAFT, a statistically consistent method for estimating species trees by maximizing quartet support on phylogenomic datasets at unprecedented scales. For n taxa and k gene trees, QRAFT achieves near-linear running time of O(kn log^2 n) when its recursion is balanced, which is within a polylogarithmic factor of the O(kn) time required simply to read the input. It also achieves asymptotically optimal O(kn) memory, so its memory requirements grow only linearly with the input size. QRAFT infers a million-taxon species tree from 1,000 genes in just three hours using only 298 GB of RAM, 64 CPU threads, and no GPUs. We estimate that the same analysis with ASTRAL-MP, a scalable parallel implementation of the widely used method ASTRAL, would take about 36 years (>100,000 times longer) and require prohibitive amounts of memory, far beyond practical computational limits. QRAFT maintains accuracy comparable to, and in many cases better than, leading species tree estimation methods. Notably, QRAFT reconstructed a microbial tree of life comprising 199,330 genomes in just a few hours using modest CPU resources. Thus, QRAFT brings highly accurate statistically consistent coalescent-based species tree inference to the unprecedented million-taxon regime and enables phylogenomic analyses towards the scale of the tree of life. QRAFT is publicly available at https://github.com/risenfromashes/qraft.