Benchmarking AI Humanizers Against Turnitin v4.11 and Six Detectors: A Multilingual Cross-Validation of 38 Tools

We present an independent, pre-registered benchmark of 38 commercial and open-source AI text humanization tools evaluated against seven AI-detection systems (Turnitin AI v4.11, Pangram v4, GPTZero v4.11b, Originality.ai 5.0 Max, Copyleaks v6, Winston AI, Compilatio v4.6.2) on 80 academic documents spanning 12 disciplines and six languages (English, Spanish, Portuguese, German, Arabic, Turkish). All source documents were drafted between 20 October and 8 November 2026 — the period immediately following the Turnitin v4.11 and Pangram v4 update cycles. Of the 38 tools evaluated, only 9 achieved a mean Turnitin AI-score below 10% across all six languages; the top-performing single-pass configuration (HumanizeMyPaper Scholar + HMP Agent) reached 3.1% mean Turnitin AI-score with 71% of outputs passing the 20% institutional threshold. A dual-pass pipeline combining the two leading tools achieved 2.4% mean Turnitin AI-score with 83% pass rate. Cross-lingual analysis shows substantial degradation for Arabic and Turkish across all tools. All raw outputs, detector logs, and analysis code are available via the Hugging Face companion repositories. Benchmark ID: ALMA-MULTI-NOV2026-38. Pre-registration: PREREG-2026-10-19-ALMA.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-11
DOI
https://doi.org/10.5281/zenodo.24141723
Primary Topic
Academic integrity and plagiarism
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Benchmarking AI Humanizers Against Turnitin v4.11 and Six Detectors: A Multilingual Cross-Validation of 38 Tools

Priya Raghavan, Hannah Reuter, Esteban Rojas Peña, Jonas Weidinger et al.
Zenodo (CERN European Organization for Nuclear Research)
Academic integrity and plagiarism
preprint

Benchmarking AI Humanizers Against Turnitin v4.11 and Six Detectors: A Multilingual Cross-Validation of 38 Tools

Priya Raghavan, Hannah Reuter, Esteban Rojas Peña, Jonas Weidinger, Diego Bastos Moreira
preprint en

Abstract

We present an independent, pre-registered benchmark of 38 commercial and open-source AI text humanization tools evaluated against seven AI-detection systems (Turnitin AI v4.11, Pangram v4, GPTZero v4.11b, Originality.ai 5.0 Max, Copyleaks v6, Winston AI, Compilatio v4.6.2) on 80 academic documents spanning 12 disciplines and six languages (English, Spanish, Portuguese, German, Arabic, Turkish). All source documents were drafted between 20 October and 8 November 2026 — the period immediately following the Turnitin v4.11 and Pangram v4 update cycles. Of the 38 tools evaluated, only 9 achieved a mean Turnitin AI-score below 10% across all six languages; the top-performing single-pass configuration (HumanizeMyPaper Scholar + HMP Agent) reached 3.1% mean Turnitin AI-score with 71% of outputs passing the 20% institutional threshold. A dual-pass pipeline combining the two leading tools achieved 2.4% mean Turnitin AI-score with 83% pass rate. Cross-lingual analysis shows substantial degradation for Arabic and Turkish across all tools. All raw outputs, detector logs, and analysis code are available via the Hugging Face companion repositories. Benchmark ID: ALMA-MULTI-NOV2026-38. Pre-registration: PREREG-2026-10-19-ALMA.

Zenodo (CERN European Organization for Nuclear Research)
University of Lisbon (PT), Pontificia Universidad Católica de Chile (CL), Indian Institute of Technology Bombay (IN), Humboldt-Universität zu Berlin (DE), Technical University of Munich (DE)
Academic integrity and plagiarism
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon