AI text detection in dentistry: a comparative analysis across generative models

BACKGROUND: Robust screening for AI-generated scientific text is increasingly required by journals, yet detector performance on full-length manuscripts remains unclear. Six widely used detectors were benchmarked on dentistry manuscripts, and performance was compared across state-of-the-art generators. METHODS: A total of 120 manuscripts was assembled in four groups (GPT-4.5, GPT-4o, DeepSeek-R2, and human-written; n = 30 each). Aidetector, GPTZero, Copyleaks, Originality.AI, Turnitin, and DetectingAI produced 0-100% scores. Discrimination was assessed with ROC/AUC and DeLong tests with Bonferroni correction. A pre-specified 60% threshold yielded document-level classifications. Inter-detector agreement was quantified with Cohen's κ. RESULTS: Four detectors showed high discrimination; pairwise AUC differences among these were not significant after Bonferroni correction and each outperformed Turnitin and DetectingAI. Using the 60% cut-off, sensitivities and specificities were high, with no false positives observed among the 30 human texts included in this sample. Agreement was almost perfect among the high performers, slight for Turnitin, and none for DetectingAI. Across tools, DeepSeek-R2 texts were the easiest to detect. CONCLUSIONS: On full dentistry manuscripts, four detectors showed high discrimination, whereas Turnitin showed moderate performance and DetectingAI was ineffective. A percentage-based 60% decision threshold provided reproducible, manuscript-level calls. CLINICAL RELEVANCE: These results may help dental editors and reviewers compare detectors for preliminary screening of AI-generated text under controlled conditions.

Authors

Institutions

Publication Details

Journal
Research Integrity and Peer Review
Published
2026-06-09
DOI
https://doi.org/10.1186/s41073-026-00228-9
Primary Topic
Artificial Intelligence in Healthcare and Education
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

AI text detection in dentistry: a comparative analysis across generative models

Daniele Garcovich, Jacopo Villa, Milagros Adobes Martin, Luca Lombardo et al.
Research Integrity and Peer Review
Artificial Intelligence in Healthcare and Education
article

AI text detection in dentistry: a comparative analysis across generative models

Daniele Garcovich, Jacopo Villa, Milagros Adobes Martin, Luca Lombardo, Giuseppe Siciliani
article en

Abstract

BACKGROUND: Robust screening for AI-generated scientific text is increasingly required by journals, yet detector performance on full-length manuscripts remains unclear. Six widely used detectors were benchmarked on dentistry manuscripts, and performance was compared across state-of-the-art generators. METHODS: A total of 120 manuscripts was assembled in four groups (GPT-4.5, GPT-4o, DeepSeek-R2, and human-written; n = 30 each). Aidetector, GPTZero, Copyleaks, Originality.AI, Turnitin, and DetectingAI produced 0-100% scores. Discrimination was assessed with ROC/AUC and DeLong tests with Bonferroni correction. A pre-specified 60% threshold yielded document-level classifications. Inter-detector agreement was quantified with Cohen's κ. RESULTS: Four detectors showed high discrimination; pairwise AUC differences among these were not significant after Bonferroni correction and each outperformed Turnitin and DetectingAI. Using the 60% cut-off, sensitivities and specificities were high, with no false positives observed among the 30 human texts included in this sample. Agreement was almost perfect among the high performers, slight for Turnitin, and none for DetectingAI. Across tools, DeepSeek-R2 texts were the easiest to detect. CONCLUSIONS: On full dentistry manuscripts, four detectors showed high discrimination, whereas Turnitin showed moderate performance and DetectingAI was ineffective. A percentage-based 60% decision threshold provided reproducible, manuscript-level calls. CLINICAL RELEVANCE: These results may help dental editors and reviewers compare detectors for preliminary screening of AI-generated text under controlled conditions.

Research Integrity and Peer Review
Universitat de València (ES), University of Ferrara (IT)
Gender equality
Openalex Percentile: Top 10%
Artificial Intelligence in Healthcare and Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.