A Case Study on User Perception of Parameterized LLM-Generated Narratives

Storylet-based narrative systems offer a flexible structure for interactive storytelling, but authoring a diverse set of meaningful storylets requires significant design effort. Large language models (LLMs) may have the potential to alleviate this burden by generating short-form narratives in response to predefined structural constraints for use in such systems. In this paper, we conduct a case study on ChatGPT 4o's ability to generate narrative fragments with parameterized intensity levels, and assess the alignment of user perceptions with these intended intensities. We hypothesize that newer large language models such as GPT-4o are able to effectively generate short-form narratives in response to parameterized inputs, and that user assessment of these LLMs' output will largely be in line with their intended intensities. To test this, we generate fifty stories across five unique story domains, each varying in a single parameter relevant to the story, such as success, damage to a relationship, chaos caused, or others, on a scale of 1 to 10. In a user study, participants were asked to compare pairs of stories with varying intensity differences (1,2, and 3) within each story domain. Analysis of user feedback, gathered through Likert-scale surveys, indicates a strong correlation between user-perceived intensity and the intended intensity ordering of generated stories. Additionally, user certainty in comparisons increased as the intensity gap between story pairs increased. The results support our hypothesis that LLMs can effectively generate stories with discernible intensity variations in response to parameterized input, and that users can reliably perceive these variations as intended. This research contributes to understanding the potential of LLMs in controlled narrative generation and provides support for their use in LLM-aided, hybrid narrative generation systems or games in the future.