Assessing the Reliability of Performance-Based Evaluation in Fashion Education: A Generalizability Theory Approach

This study addresses the issue of score inconsistency in performance-based assessment (PBA), particularly in creative disciplines where fairness and reliability are crucial. The main objectives of this study were to evaluate the reliability of PBA scores and analyze sources of measurement error using Generalizability Theory (G-Theory). This quantitative study employed a fully crossed (p × r × d) design in an eveningwear construction course. A total of 70 student fashion products were assessed by four assessors consisting of two internal lecturers and two external industry professionals byusing a five-dimensional rubric. Data were collected through standardized scoring sheets and analyzed using linear mixed-effects modeling in R software (lme4 package). Variance components were estimated in the Generalizability Study (G-Study), followed by simulations in the Decision Study (D-Study) to test various assessment configurations. The G-Study results indicated that the largest variance component came from residual error (50.42%), followed by product differences (43.44%), while the contributions of assessors and dimensions were relatively small. The D-Study showed that increasing the number of raters and rubric dimensions significantly increased reliability; a configuration of four raters and five dimensions resulted in a generalizability coefficient of 0.9185. These findings confirm that G-Theory is effective for diagnosing and optimizing reliability in the context of creative assessment. This research provides practical insights for educators in designing fairer and more reliable assessment systems, and opens up opportunities for the application of psychometrics in vocational and design education.