How Text Transformations Affect AI Detection

This study addresses the critical issue of AI writing detection, which currently plays a key role in deterring technology misuse and proposes a foundation for the controllable and conscious use of AI. The ability to differentiate between human-written and AI-generated text is crucial for the practical application of any policies or guidelines. Current detection tools are unable to interpret their decisions in a way that is understandable to humans or provide any human-readable evidence or proof for their decisions. We assume that there should be a traceable footprint in LLM-generated texts that is invisible to the human eye but can be detected by AI detection tools-referred to as the AI footprint. Understanding its nature will help bring more light into the guiding principles lying at the core of AI detection technology and help build more trust in the technology in general. The main goal of this paper is to examine the AI footprint in text data generated by large language models (LLMs). To achieve this, we propose a new method for text transformation that should measurably decrease the AI footprint in the text data, impacting AI writing scores. We applied a set of stage-by-stage text transformations focused on decreasing meaningfulness by masking or removing words. Using a set of AI detectors, we measured the AI writing score as a proxy metric for assessing the impact of the proposed method. The results demonstrate a significant correlation between the severity of changes and the resulting impact on AI writing scores, highlighting the need for developing more reliable AI writing identification methods that are immune to attempts to hide the AI footprint through subtle changes