Replication and methodological robustness in quantitative typology

Abstract Replication and replicability are fundamental tools to ensure that research results can be verified by an independent third party, reproducing the original study and ideally finding similar results. Yet, replication has not played a very important role in language typology so far, with most of the discussion around replication concerned with different types of language samples and sampling methods. This study addresses the issue of replication in typology in a different way. We use the original datasets of four previous typological studies (Berg 2020; Dryer 2018; Seržant 2021; Shcherbakova et al. 2023) to show how statistical modeling can be used to test methodological robustness in typology. We do so employing advanced statistical bias controls, namely phylogenetic regression for genetic effects and a Gaussian Process for contact effects. While we could replicate some of the original results, parts of our findings differed from the original ones, revealing important methodological insights. Our comparisons show that more advanced statistical techniques that can model the phylogenetic and contact relations between languages pick up more complex patterns in the data than traditional sampling methods, and they capture more of the real relations between languages and their effects on linguistic structure.