Prompt Override: LLM Hacking as Serious Game

In this demo paper we present Prompt Override, a serious game in which players engage in prompt-based hacking challenges by manipulating the system prompt of a large language model (LLM) to solve puzzles. The game features two LLMs: one as the target of the player's hacking attempts, and another as a rogue assistant guiding the player throughout the game. As players explore a simulated file system, editing prompt snippets, Prompt Override offers a unique take on the hacking game genre-one where rigid solution paths are replaced with more open-ended, language-driven problem solving. By leveraging realworld limitations and vulnerabilities of LLMs, Prompt Override encourages players to develop a deeper understanding of prompt engineering, model behavior, and the ethical implications of interacting with intelligent systems.