Identify user features impacting keystroke, mouse and touchscreen dynamics

Abstract To understand how individual user features affect one’s emotional changes in response to stimuli or changes in a situated external environment, it is necessary to measure how user features affect one’s behavioral changes (patterns of changes) in the use of keyboard, mouse, and touchscreen devices (KMT dynamics), as the measurable behavioral changes are related to both user features and the user’s emotional states. Our central research questions are: (1) which user features have a significant impact on behavior such as KMT dynamics? (2) how do these user features impact behavior from qualitative and quantitative perspectives? and (3) what is an effective framework to support personalized emotion recognition? To answer these questions, we conducted a questionnaire survey with 107 participants to investigate how, and to what extent, user profile features can influence keyboard, mouse, and touchscreen usage in dynamics terms (KMT dynamics). We then performed data analysis to explore the relationships between an initial set of user feature candidates and an initial set of KMT dynamics feature candidates (12 identified from a literature review). Based on the identified relationships, we identified seven important user features and seven important KMT dynamics features for further KMT-based emotion recognition studies. The seven key user features are: gender, KMT proficiency, language proficiency, hand using habit, software proficiency, task urgency, and emotion, while the seven important KMT dynamics features are: typing speed, typing error rate, pressure value of key/strength of pressing touchscreen, mouse (finger or pencil) movement speed, clicking speed, clicking strength, and keystroke speed. Finally, we proposed a new emotion recognition framework incorporating the identified key user and KMT dynamics features as a guiding model for further research in this field.