Does Graph Structure Improve AI-Generated Text Detection? A Controlled Audit with Tuned Baselines

Abstract Graph neural network (GNN) detectors of AI-generated text report state-of-the-art accuracy, and the architectural claim that graph topology captures signals invisible to sequential classifiers is now common in the literature. We report the complete lifecycle of one such detector, from construction to falsification. AEGIS, a graph attention network over k-nearest-neighbour text graphs with multi-signal edge weights, reaches 98.50% accuracy on scientific abstracts with only 73K trainable parameters, outperforming five established baselines. We then subject our own system to the evaluation its headline number deserves: generation-time length control, format normalization, leakage-free splits, a tuned frozen-embedding probe as the calibrated null, and a 10 × 10 cross-generator transfer study over three seeds. Under this protocol the graph's apparent contribution collapses from +0.240 AUROC over a surface baseline to −0.0004 AUROC against the tuned probe, inside the measured replication noise floor, and the two detectors' transfer geographies are near-identical (ρ = 0.986). A single untuned regularization default manufactured a +0.027 phantom advantage. The structural differences the graph encodes are real, but they are a re-encoding of information already present in the embeddings. We distil the audit into a practical checklist for evaluating architectural claims in text detection, and argue that accuracy numbers and mechanism claims must be validated separately.

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-12
DOI
https://doi.org/10.21203/rs.3.rs-11039287/v1
Primary Topic
Authorship Attribution and Profiling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Does Graph Structure Improve AI-Generated Text Detection? A Controlled Audit with Tuned Baselines

Meriyem Chergui, Abdelkarim Ammoumou, Abadila Alaktif, Gmira Faiq
Research Square
Authorship Attribution and Profiling
preprint

Does Graph Structure Improve AI-Generated Text Detection? A Controlled Audit with Tuned Baselines

Meriyem Chergui, Abdelkarim Ammoumou, Abadila Alaktif, Gmira Faiq
preprint en

Abstract

Abstract Graph neural network (GNN) detectors of AI-generated text report state-of-the-art accuracy, and the architectural claim that graph topology captures signals invisible to sequential classifiers is now common in the literature. We report the complete lifecycle of one such detector, from construction to falsification. AEGIS, a graph attention network over k-nearest-neighbour text graphs with multi-signal edge weights, reaches 98.50% accuracy on scientific abstracts with only 73K trainable parameters, outperforming five established baselines. We then subject our own system to the evaluation its headline number deserves: generation-time length control, format normalization, leakage-free splits, a tuned frozen-embedding probe as the calibrated null, and a 10 × 10 cross-generator transfer study over three seeds. Under this protocol the graph's apparent contribution collapses from +0.240 AUROC over a surface baseline to −0.0004 AUROC against the tuned probe, inside the measured replication noise floor, and the two detectors' transfer geographies are near-identical (ρ = 0.986). A single untuned regularization default manufactured a +0.027 phantom advantage. The structural differences the graph encodes are real, but they are a re-encoding of information already present in the embeddings. We distil the audit into a practical checklist for evaluating architectural claims in text detection, and argue that accuracy numbers and mechanism claims must be validated separately.

Research Square
University of Hassan II Casablanca (MA)
Authorship Attribution and Profiling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon