An Assessment of Construct Validity and Nomological Validity of the Measurement Model for ChatGPT UX Evaluation

The present study aims to evaluate the construct validity and nomological validity of the measurement model for ChatGPT UX evaluation. A careful literature review leaded to the identification of a total of thirty-six UX evaluation metrics. N=16 respondents participated in Delphi and nine ChatGPT UX evaluation metrics were selected, including performance expectancy, effort expectancy, usability, capability, communication, co-creation, effectiveness, innovativeness, and interaction quality. A survey study was conducted and N=212 respondents rated their ChatGPT UX via fifty-six measurement items. An exploratory factor analysis was performed on the collected survey data and the result indicates that two UX metrics of usability and effectiveness were excluded. A confirmatory factor analysis was conducted to assess the construct validity of the measurement model, and the results reveal that the validity level of the measurement model proposed in the present work is fairly satisfactory. Lastly, the results from the test of nomological validity of the measurement model indicate that, the seven UX evaluation metrics collectively constitute a single second-order construct representing ChatGPT UX.