Constructing a multimodal feature set for pain intensity classification

Abstract Previous approaches to pain intensity classification have typically relied on small sets of top-performing features to maximize accuracy. While effective in constrained scenarios, such strategies neglect the diverse range of modalities available in modern pain databases. In this work, we introduce a multimodal feature set (MMFS) that integrates heterogeneous features from each biosignal modality in the BioVid and X-ITE databases. Our approach captures a broad spectrum of complementary information, maintaining robustness even when individual modalities are unavailable. Experimental results show consistent performance improvements, with classification accuracy increasing by up to 8% overall and by 6% when evaluating all pain levels. Through detailed analyses of individual modalities, confusion matrices, and a modality ablation study, we demonstrate that the combined effect of multimodality and balanced information distribution drives these gains. Furthermore, feature importance analysis reveals which inputs contribute most to final predictions and which features are most beneficial for the more challenging pain classes.