Machine learning classification of active viewing of pain and non-pain images using EEG does not exceed chance in external validation samples

Previous research has demonstrated that machine learning (ML) could not effectively decode passive observation of neutral versus pain photographs by using electroencephalogram (EEG) data. Consequently, the present study explored whether active viewing, i.e., requiring participant engagement in a task, of neutral and pain stimuli improves ML performance. Random forest (RF) models were trained on cortical event-related potentials (ERPs) during a two-alternative forced choice paradigm, whereby participants determined the presence or absence of pain in photographs of facial expressions and action scenes. Sixty-two participants were recruited for the model development sample. Moreover, a within-subject temporal validation sample was collected, consisting of 27 subjects. In line with our previous research, three RF models were developed to classify images into faces and scenes, neutral and pain scenes, and neutral and pain expressions. The results demonstrated that the RF successfully classified discrete categories of visual stimuli (faces and scenes) with accuracies of 78% and 66% on cross-validation and external validation, respectively. However, despite promising cross-validation results of 61% and 67% for the classification of neutral and pain scenes and neutral and pain faces, respectively, the RF models failed to exceed chance performance on the external validation dataset on both empathy classification attempts. These results align with previous research, highlighting the challenges of classifying complex states, such as pain empathy using ERPs. Moreover, the results suggest that active observation fails to enhance ML performance beyond previous passive studies. Future research should prioritise improving model performance to obtain levels exceeding chance, which would demonstrate increased utility.