Design principles for generating real-time explanations to learners’ questions in a virtual reality learning environment using natural language generation: Practice based lessons learned from creating multiple agents in Unreal Engine 5 using ConvAI

The effectiveness of a virtual reality learning environment (VRLE) can be enhanced through adaptive instruction where pedagogical agents adjust their instruction to the learner’s actions. Despite advances in natural language processing, the lack of natural communication of pedagogical agents in a VRLE is reducing its effectiveness. Furthermore, there is a lack of design principles for developing pedagogical agents that respond intelligently to learners and can converse naturally. This paper is part of a larger study that investigates the design principles for effective pedagogical agents in VRLEs that provide real-time explanations using natural language generation. It reports on the lessons learned while developing a pedagogical agent using Unreal Engine and ConvAI. Two key design principles were identified, namely the importance of multiple pedagogical agents with distinct roles and well-rounded personas. A pedagogical agent was developed, and initial tests show that the agent responded intelligently in an appropriate context.