Can AI extort like humans do? Understanding the construction of illicit genres by large language models vs humans

Abstract This paper presents a linguistic contribution to the growing literature on the use of large language models (LLMs) for malicious purposes. Specifically, we draw on genre theory and methods to investigate the comparative construction of illicit genres by LLMs and humans, focusing on commercial extortion notes. We investigate the most effective tools and prompts for generating extortion notes, examine how closely LLMs are currently able to reproduce human-authored extortion notes, and consider how different LLMs compare in their production capabilities. These questions are explored using 24 prompts across seven thematic categories tested on five LLMs (ChatGPT-4, Claude 3.5 Sonnet, Gemini 2.0, Copilot, and Llama 3.2 3B), and analysis of 36 resulting texts using Swales’ (1990) rhetorical moves framework. Finally, we compare our results with findings from Petykó et al. (2025) on human-authored extortion notes. Key findings reveal that prompts involving fictional scenarios and simple example requests were most effective, with Llama, ChatGPT, and Gemini producing the most successful outputs. LLM-generated notes largely shared the same communicative functions as human-authored ones but exhibited distinct features, including two additional moves (‘Administrative’ and ‘Inviting cooperation’) and higher frequencies of formal elements. Different LLMs showed subtle but identifiable patterns in their outputs, suggesting potential for attribution. These findings have implications for law enforcement in identifying LLM-assisted crime, technology developers in improving ethical safeguards, and linguistic researchers in understanding variation in genre production.