Large-scale benchmarking of open gender-inference systems in scholarly identity data

This preprint presents a large-scale benchmark of open name-based gender-inference methods in scholarly identity data. Using an ORCID-derived reference set of 330,994 authors, the study compares NomQuamGender, WGND-based approaches and a hierarchical WGND-based pipeline. The evaluation examines coverage, precision, confidence thresholds, name preprocessing, cultural rules, regional differences and agreement between systems. The findings illustrate the trade-offs between coverage and precision, identify contexts in which classification errors are more frequent, and examine how confidence thresholds and abstention strategies can improve the reliability of gender-inference procedures used in bibliometric research. Preprint status: This manuscript has been submitted to the Journal of Big Data and has not yet undergone peer review. Supplementary materials: The aggregate results, supplementary tables, verification code and accompanying documentation are available in a separate Zenodo record: https://doi.org/10.5281/zenodo.23842331. No individual-level author records are distributed with this preprint. Funding: These materials are part of project PID2024-156739NB-I00, funded by MCIU/AEI/10.13039/501100011033 and by ERDF, EU.