AI-based User Emotion Recognition from Interaction Data: Challenges and Guidelines for Training Data Creation

Artificial Intelligence (AI) is a rising topic in the field of emotion recognition, e.g., from facial expressions. However, existing methods often require to be performed in staged set ups and are obtrusive by gathering additional data. Especially, collecting video data includes a high data protection risk. Our approach is to provide an unobtrusive emotion recognition tool based on Keystroke, Mouse and Touchscreen (KMT) data. Recently, we published a data set for emotion recognition from keystroke and mouse interaction data. In this paper, we present the challenges we faced during the creation of the data set. This covers collecting User Interface (UI) data as well as emotional ground truth data. For each of seven mentioned challenges, we provide our solutions as well as guidelines for other researchers to prevent them. The challenges include possible issues with recorded data as well as issues of automated facial coding engines. We provide a possible approach for manual facial coding and describe aspects attention should be paid to. Furthermore, we indicate issues when using different software tools to collect the data. The paper aims to help other researchers by providing insights and a guideline for the creation of the data set. We make these insights available for other researchers who want to create similar data sets or who want to expand ours. By sharing our insights, we aim to improve the reproducibility of AI training data creation and AI-based emotion recognition from user interaction data.