An <scp>RL</scp> ‐Driven Adaptive Game Approach to Support Cultural Heritage Learning

ABSTRACT Background Study Serious games for cultural heritage offer opportunities for enhancement, particularly in user experience and educational impact. This paper presents a Reinforcement Learning (RL)‐driven adaptive approach to support CH learning through a graphic adventure game. Objectives The proposed solution integrates an artificial intelligence service based on the Expected State‐Action‐Reward‐State‐Action (SARSA) algorithm to provide dynamic, timely, and personalised in‐game support. This service adjusts the Level of Detail (LoD) in the assistance offered to players, promoting an engaging learning experience tailored to individual needs and pacing. Methods The educational game, ForumSG, reconstructs the Roman Forum of Thessaloniki, offering exploratory learning within a 3D virtual environment. An in‐game activity—the lost‐wax method of coin production—was used to evaluate the adaptive service with both simulated and real users. The game collects user behavior data and uses the RL service to determine when and how to provide support, for an optimal balance between challenge and skill. Results Experimental testing with synthetic users showed that the Expected SARSA agent successfully learned behaviorally consistent action‐selection policies, favouring lower LoD in uniform‐probability settings and higher LoD in more complex states, driven by reward function and transition efficiency. A preliminary pilot assessment with real users validated the functionality of the RL system. The game was well received, with participants appreciating the timely and supportive guidance, exploratory nature of the experience and its cognitive engagement. Conclusion This work contributes to the field by combining pedagogical, museological, and technological design principles in Cultural Heritage Serious Games. The developed RL architecture is lightweight compared to deep learning methods, allowing for easy integration into other educational games, and providing a foundation for scalable, personalised CH learning systems.