A Multimodal Approach to User Experience Evaluation: Integration of Physiological, Behavioral, Contextual and Self-Report data with UserSence

User experience (UX) evaluation provides feedback that contributes to the understanding of the impact of design elements on the experience. Traditional UX evaluation methods are based on the participants’ subjective input, as users’ responses shape the evaluation insights. Meanwhile, user experience evaluation procedures include physiological measurements as feedback to assess an experience. These data are considered more objective due to the absence of complete control of bodily responses, and allowing the measurement of physical and emotional states. Integrating qualitative and quantitative data with multiple physiological responses through triangulation shapes a multimodal user experience evaluation approach that offers cross-checking insights, avoids relying on a single evaluation method, provides more credibility to the findings, and integrates different pieces of information, leading to a comprehensive understanding of the experience. This paper aims to present UserSence, a user experience evaluation tool that assembles information to identify users’ affective states, enabling evaluators to gather data for analysis and decision-making. We present our findings from two pilot studies and conclusions provide further insight into the opportunities and pitfalls of UX evaluation based on multimodal input and the associated practical and ethical implications.