Audience evaluation of heritage-derived cultural-creative products: a multilevel analysis of rater-level and product-level variance

Abstract Studies of heritage-derived cultural-creative products often interpret audience ratings as differences between products, yet each rating also reflects the person who gives it. This study examined how these two sources of variance contribute to evaluations of six designs derived from the Wa jicong top, a spinning top from Yunnan, China. Each of 228 respondents rated two randomly assigned designs, giving 456 evaluations. Crossed random-effects models estimated the rater and product shares of each response, and within-rater and between-rater analyses separated two kinds of association among evaluative judgements. Raters accounted for 81.6% of the variance in design perception, 81.8% in perceived cultural authenticity and 56.1% in aesthetic evaluation. Average differences between products accounted for less than 1% of each judgement; these shares were similar under alternative sample screenings. Cultural familiarity, measured through self-rated knowledge of the source culture and interest in ethnic styling, explained 7.2% to 22.8% of rater-level variance. The ranking of products on authenticity also varied with familiarity. Of the descriptive perceptions, dynamism showed the clearest product difference (product share 4.9%), although its observed ordering did not follow the designs' intended static and dynamic forms. The three evaluative judgements were strongly associated between raters. Within raters, no association between authenticity and aesthetic evaluation was detected, including after correction for measurement error. The indirect association of design perception with aesthetic evaluation through authenticity was 0.31 between raters and 0.00 within them. In this design system, evaluative scores primarily distinguished audience members. Repeated-stimulus studies should report how variance divides between raters and products and distinguish within-rater from between-rater associations before interpreting pooled coefficients.