Toward a more discriminative and culturally responsive TALIS teacher self-efficacy in multicultural classrooms scale: cross-national evidence from Japan and Colombia using the generalized partial credit model

Abstract Background The Teacher Self-Efficacy in Multicultural Classrooms (TSMC) scale, included in the Teaching and Learning International Survey (TALIS) 2018, aims to measure teachers’ confidence in handling culturally diverse classrooms. However, its cross-cultural psychometric properties remain underexplored. This study evaluates the structural validity of the TSMC scale, focusing on its performance across different cultural and educational contexts. Methods As a preliminary analysis, a clustering procedure was conducted on 34 countries, excluding those with excessive missing rates from the original 47 TALIS participants. Based on the results, Japan and Colombia, which represented the two extreme ends, were selected for further investigation using the Generalized Partial Credit Model (GPCM). In particular, Japan and Colombia represent two educationally and culturally contrasting contexts with differences in curriculum flexibility, institutional practices, assessment approaches, and demographic composition. After confirming the assumption of unidimensionality, this study evaluated the structural validity of the TSMC scale by examining possible floor and ceiling effects, category response curves, test information curves, and person–item maps. Results The analysis revealed pronounced ceiling effects in Colombia and floor effects in Japan, highlighting the scale’s limited capacity to discriminate among high- and low-ability respondents. Category response curves further illustrated that the middle response options failed to capture subtle differences in teacher self-efficacy, while test information curves and person-item maps indicated biased theta values, highlighting a misalignment between the instrument's theoretical structure and its practical application, thereby compromising its structural validity. Conclusions To address the identified limitations, the present study suggests revising the TSMC scale by expanding theoretical depth, enhancing measurement sensitivity, pilot testing & iteration, and considering countries’ sociocultural differences for a more accurate interpretation of the results. The proposed refinements could enhance the scale’s validity and cross-cultural applicability, ensuring culturally responsive and psychometrically sound assessments of teachers’ multicultural self-efficacy. The recommendations hold implications for improving large-scale educational surveys and informing policy interventions to support culturally responsive teaching worldwide.