Video-based contactless detection of task-related concentration using advanced machine-learning techniques

The present study aimed to test the accuracy of applying machine learning to a novel contactless video-based approach in detecting task-related concentration. Evaluations of concentration on-task have relied on laboratory methodologies, which encounter difficulties when applied to real work scenarios. Video photoplethysmography (VPPG) can present a solution to these difficulties by extracting physiological changes from videos captured by any conventional camera. Applying machine learning to physiological signals from VPPG can enable contactless detection of task-related concentration. Thirty adults completed a simulated task. Physiological changes were recorded via electrocardiogram (ECG) and VPPG. Pre-trained VGG, support vector machine, and XGBoost were performed on ECG and VPPG signals to detect when participants were on- or off-task. The ensemble method, which combined three machine-learning methods, applied to VPPG signals proved to be highly accurate (∼97%). Among individual machine-learning methods, pre-trained VGG applied to VPPG signals performed the best, comparable to the ensembled method. All analyses showed detection based on VPPG signals to significantly outperform ECG signals. Results establish a proof-of-concept that VPPG and machine learning can be used to detect task-related concentration in a contactless, convenient, and inexpensive fashion. VPPG can enable the detection of task-related concentration in natural work settings.