From Facial Expressions to User Experience (UX): How User Emotions Define the Design of Intelligent Systems

Emotions are a key determinant of User Experience (UX). This work investigates the relationship between facial emotions and UX, presenting <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ALPHA</i>, a predictive system that estimates <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">User Satisfaction</i> based on valence–arousal (VA) signals. The <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Dataset DT22</i>, a database of 112 participants, is used to evaluate models on the external test set. An <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">identity-free</i> processing pipeline that uses only VA features achieves ∼77% accuracy/weighted <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</i><sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub>-score. When context information is also available—Movie Clip ID and perceived <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">User Quality</i>—the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">context-aware</i> pipeline increases performance to ∼85–92% in ablation analyses, by using ensembles with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">majority voting</i> decision rule. This indicates that contextual features provide complementary information to VA. To assess system robustness, Leave-One-Participant-Out (LOPO) and Leave-One-Movieclip-Out (LOMO) protocol schemes, as well as emotion-anchored evaluations (first- and last-emotion), are investigated. The approach is modular and adheres to privacy-by-design principles by operating on identity-free descriptors rather than storing facial frames. Overall, the findings suggest that emotion signals can support adaptive UX and enable dynamic content personalization in applications such as video streaming, customer-experience management, and e-learning. This research presents a practical study on integrating emotion-aware intelligence while respecting user privacy.