Hyper-Realistic Digital Twin of a Coach: Development and Evaluation of a Sports Training Simulator

Thanks to advances in the field of computer graphics and natural language processing, it is increasingly possible to create multimedia applications that use a hyper-realistic avatar as the main element of the communication interface with the user. However, major challenges remain, especially when the avatar is required to simulate the appearance and behavior of a real person. This work describes the methodology and tools necessary to create a hyper-realistic avatar that simulates the appearance and behavior of a real person. The methods and tools have been tested by developing a new application that simulates the sports training process. The design of the new application is based on the hypothesis that a digital system that hyper-realistically simulates real training can offer similar benefits to the real one and, therefore, be used to complement real training sessions and thus facilitating the continued performance of the exercise. In this work, a complete sports training simulator has been developed with special emphasis on simulating the coach's real appearance, his way of moving and speaking, as well as a personalized exercise routine generator. Finally, a between-subjects study was conducted comparing sessions led by a real coach versus his hyper-realistic digital twin. Twelve people participated in two 15-minute sessions, one with each type of coach. The analysis was performed using user experience questionnaires and physiological metrics captured through tracking bands during training sessions. The questionnaires indicated that users positively value the system as a complementary tool to their usual training routines. On the other hand, physiological metrics revealed that while users initially exert less effort with the digital avatar, their performance levels, in terms of HRV and temperature, tended to align with those of the in-person sessions halfway through the training. These findings provide valuable insights into the effectiveness and impact of digital avatars compared to real coaches in the context of fitness training.