Emotion recognition from auditory autonomous sensory meridian response (ASMR) using multi-modal physiological signals

<i>Objective.</i>Autonomous sensory meridian response (ASMR) is a tingling sensation induced while attending to specific sounds, including whispering, tapping, scratching, or other soft, repetitive noises. While previous studies focused on low arousal-positive emotions such as relaxation and calmness, this study explores a broader range of emotions elicited by ASMR auditory stimuli, including happiness, sadness, and disgust.<i>Approach.</i>The proposed study collects the multi-modal physiological data from electroencephalography, photoplethysmography, and electrodermal activity via wearable bio-sensors from 23 ASMR-experiencing participants while exposed to different ASMR-inducing auditory stimuli. It employs the rmANOVA test on the collected physiological responses and self-reported ratings for quantitative analysis and results in a significant difference between the emotions induced from the four audio stimuli, i.e. Happy from A1, Sad from A2, Calm from A3, Disgust from A4, and the neutral state. The proposed study also applies deep learning classifiers, artificial neural network (ANN), and convolution neural network (CNN) to the collected multi-modal physiological data to classify the four induced emotions from the ASMR auditory stimuli using the dimensions of arousal, valence, and dominance.<i>Main results</i>. The classification accuracy results from ANN, and CNN prove an excellent success rate of 96.12% and 74.25% with multi-modal valence-arousal-dominance for ANN and CNN, respectively, in classifying the four emotions induced by ASMR stimuli. And the statistical rmANOVA test results indicated distinctions among the four emotions, as the<i>p</i>-values exceeded the significance threshold of 0.05.<i>Significance.</i>The results highlight the effectiveness of multi-modal physiological signals and deep learning in reliably classifying ASMR-induced emotions, contributing to advancements in emotion recognition for mental health and therapeutic applications.