A Novel Benford’s Law-Driven Approach for Detecting Machine-Generated Text

Detecting Machine-Generated Text (MGT) is critical for the sustainable development of information systems. Existing studies often overlook the generation inconsistency between AI and human, limiting the effectiveness of detection. This article introduces a novel detection approach, BENADV, and the motivation stems from our recognition that, unlike MGT (probabilistic token prediction), Human-Written Text (HWT) is influenced by individual factors (e.g., personal experience), which constitutes a form of “manipulation” at the textual level. Specifically, BENADV is built on our new discovery that MGT adheres more closely to Benford’s law compared to HWT. We leverage the adherence patterns as detection mechanisms, and further enhance detection performance through adversarial perturbations controlled by stochastic differential equations. Extensive experiments on general-domain datasets demonstrate that BENADV is SOTA. For instance, on the HC3 dataset, BENADV achieves 99.13% accuracy and 99.18% F1, outperforming existing methods by 1.16–7.82% and 1.37–8.30%. Moreover, BENADV exhibits remarkable scalability, with its performance consistently exceeding 96% on vertical domain datasets as AI advances (from GPT-3.5 to GPT-4), far surpassing the 50–60% performance of existing methods. Notably, BENADV excels in the more challenging short MGT detection. Also, we provide practical insights and discuss implications.