Open Weight Large Language Models as a design material in RPGs

With the advance of Large Language Models in recent years, there has been a lot of interest in using them as a design material. Games with a large quantity of text, such as Role-playing Games, have been one area with a lot of interest in the gaming industry. This study aims to explore how open-weight models can perform as a design material, running locally on an end user’s machine. The exact use of the Large Language Model as a design material is as a personalization tool of a game’s dialogue scripot. This is explored through participant studies on a custom-made artefact, where participants provide data through semi-structured interviews and Likert scale metrics. Additionally, the outputs of the Large Language Model are evaluated with a simplified Natural Language Processing quality measurement protocol, allowing the researcher to also evaluate output quality, and explore links between participant data and output quality in the scope of text rewriting of game dialogue. The study’s results finds that different aspects of game dialogue benefit unequally from personalization, and that results for areas where they perform worse differ highly depending on what type of personalization the model is asked to perform. The results show that without further development or more constraints, the technology is not ready for deployment on a full game, but shows promise in specific parts of a game.