OPEN: A Benchmark Dataset and Baseline for Older Adult Patient Engagement Recognition in Virtual Rehabilitation Learning Environments

Engagement in virtual learning is critical to participant satisfaction, performance, and adherence, particularly in domains such as online education and virtual rehabilitation, where interactive communication is essential for success. However, accurately measuring engagement in virtual group settings remains a significant challenge. There is growing interest in leveraging artificial intelligence for large-scale, real-world, and automated engagement recognition. Although engagement has been widely studied among younger populations in academic contexts, research methods and datasets focused on older adults in virtual and telehealth learning environments remain limited. Existing approaches often overlook the contextual relevance of learning materials and the longitudinal dynamics of engagement across sessions. This paper presents OPEN (Older adult Patient ENgagement), a novel dataset created to support the development of artificial intelligence models for engagement recognition. The dataset was collected from eleven older adults participating in virtual group learning sessions over six weeks as part of their cardiac rehabilitation programs, yielding over 35 hours of data, which represents the largest dataset of its kind. While raw video is withheld to protect privacy, the publicly released data include facial, hand, and body joint landmarks, as well as behavioral and affective features extracted from video. Observational annotations comprise second-level binary engagement states, emotional and behavioral labels, and context-type details, such as whether the instructor addressed the group or an individual. Multiple versions of the dataset were generated using sample lengths of 5, 10, and 30 seconds, as well as variable-length segments. To demonstrate its utility, various machine learning and deep learning models were trained on the annotated data, achieving engagement recognition accuracy as high as 81%. OPEN provides a scalable foundation not only for advancing personalized, artificial intelligence-driven engagement recognition in aging populations but also for contributing to broader engagement recognition research.