Measuring AI literacy in design education: development and validation of the artificial intelligence literacy scale for design students

Abstract Generative Artificial Intelligence (AI) is increasingly embedded in design practice, challenging how design education fosters students’ AI literacy and thus prepares students to engage efficiently with AI. However, there is currently no domain-specific theoretical framework and reliable instrument in the literature to conceptualize and assess design students’ AI literacy, hindering its systematic cultivation. This study fills in this gap by developing and validating the Artificial Intelligence Literacy Scale for Design Students (AILS-DS), in accordance with established standards for scale development. The opinions of a wide range of experts were gathered through a qualitative method in order to define the AI literacy for design students, and served as a theoretical framework for the scale’s item pool. Exploratory Factor Analysis (EFA, N = 294) and Confirmatory Factor Analysis (CFA, N = 325) were then applied to explore the underlying factor structure of the scale and validate its psychometric properties. The final 23-item, five-factor scale demonstrated good reliability and construct validity. Five factors (AI Concepts and Technical Skills, AI Tool Proficiency, AI Ethics and Originality Awareness, Recognition of AI Value in Design, and Independent Evaluation of AI Outputs) in the AILS-DS capture both widely acknowledged AI literacy components and design-specific structure, reflecting the designerly way of knowing and thinking. This study provides a practical tool for educators and researchers, supporting the assessment and cultivation of AI literacy in design education and laying a foundation for pedagogical innovation in the AI era.