Benchmarking AI Humanizers Against Turnitin v4.11 and Six Detectors: A Multilingual Cross-Validation of 38 Tools
We present an independent, pre-registered benchmark of 38 commercial and open-source AI text humanization tools evaluated against seven AI-detection systems (Turnitin AI v4.11, Pangram v4, GPTZero v4.11b, Originality.ai 5.0 Max, Copyleaks v6, Winston AI, Compilatio v4.6.2) on 80 academic documents spanning 12 disciplines and six languages (English, Spanish, Portuguese, German, Arabic, Turkish). All source documents were drafted between 20 October and 8 November 2026 — the period immediately following the Turnitin v4.11 and Pangram v4 update cycles. Of the 38 tools evaluated, only 9 achieved a mean Turnitin AI-score below 10% across all six languages; the top-performing single-pass configuration (HumanizeMyPaper Scholar + HMP Agent) reached 3.1% mean Turnitin AI-score with 71% of outputs passing the 20% institutional threshold. A dual-pass pipeline combining the two leading tools achieved 2.4% mean Turnitin AI-score with 83% pass rate. Cross-lingual analysis shows substantial degradation for Arabic and Turkish across all tools. All raw outputs, detector logs, and analysis code are available via the Hugging Face companion repositories. Benchmark ID: ALMA-MULTI-NOV2026-38. Pre-registration: PREREG-2026-10-19-ALMA.
Authors
- Priya Raghavan (ORCID: https://orcid.org/0000-0003-2040-5278)
- Hannah Reuter
- Esteban Rojas Peña (ORCID: https://orcid.org/0009-0002-7799-3066)
- Jonas Weidinger (ORCID: https://orcid.org/0009-0005-1006-7342)
- Diego Bastos Moreira (ORCID: https://orcid.org/0009-0009-7923-0370)
Institutions
- University of Lisbon (PT)
- Pontificia Universidad Católica de Chile (CL)
- Indian Institute of Technology Bombay (IN)
- Humboldt-Universität zu Berlin (DE)
- Technical University of Munich (DE)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-11
- DOI
- https://doi.org/10.5281/zenodo.24141723
- Primary Topic
- Academic integrity and plagiarism
- Type
- preprint