Soft Real-time Emotion-aware System through Human Physiological Signals using Pervasive Devices

The concept of connected things that involves emotional aspects has been raised as a new research issue known as “emotional IoT”. One of the challenges is to recognize the emotion in real-time without interfering the user’s task or activity. However, most of the existing works on emotion recognition deal with high sampling rate devices and thus need complex arrangement. In this study, we present a framework to recognize four emotions based on human physiological signals using pervasive devices with low sampling rate. The employment of low sampling rate devices may degrade the classification accuracy due to the data offsets or calibration errors. Our study proposes a set of energy deviation features to overcome the accuracy degradation caused by low sampling rate devices. The recursive feature elimination method is applied to avoid overfitting by selecting the most important features and reduce the computational overhead in real-world implementation. Experimental results show that the proposed approach achieves 92.01% and 91.75% accuracy in terms of valence and arousal, respectively. The realization of the proposed approach is done in an end-to-end soft real-time emotion recognition system using smartphone and smartwatch devices. The performance of our system was evaluated under laboratory setup, resulting in 46.3 milliseconds of response time on the average for recognizing emotions.