Real-time Multi-modal Comprehensive Bodyweight Exercise Feedback System with a Personal Virtual Expert Guiding

Bodyweight exercises have numerous health benefits when performed correctly. Yet, such exercises require the supervision of a professional to prevent injuries while maximizing the health benefits. However, gaining access to an expert can be challenging due to financial burdens or scheduling concerns. In this paper, we present a real-time multi-modal exercise feedback system based on experts' data, aiming to provide bodyweight exercise guidance to users. In this context, we utilized body pose and floor-based pressure data for bodyweight squats, lunges, and crunches. After recording the exercise data of certified fitness trainers, we created our virtual experts to provide expert-aligned motion and force feedback to the users on our platform. Users were given the option to select virtual experts depending on their fitness goals, while the expert data was personalized for real-time feedback. As a result, we observed significant improvements in users' movement and force during the exercise sessions, along with greater preference and heightened motion awareness.