Advancing University Core Competency Diagnostic Tools for Enhanced Flexibility and Sustainability: A Case Study

Objectives This study aims to propose a phased advancement strategy for university core competency diagnostic tools that enhance flexibility and sustainability in responding to external and internal changes while ensuring the reliable and continuous measurement of core competency-based educational outcomes. Methods The study explores a phased enhancement strategy through a case analysis of University A, which proactively implemented competency diagnostics. The enhancement process included three stages: planning, competency structure improvement and item development, and longitudinal analysis with test equating. During the planning stage, internal and external university contexts were analyzed, and problematic items were identified based on existing diagnostic results (2020-2023). Preliminary items were developed through literature review and comprehensibility testing, then refined through expert consultations. A pilot and main survey validated construct validity and reliability, leading to final item selection. Post-enhancement, test equating was applied to ensure comparability of pre- and post-enhancement results, followed by mean score changes and LGM analysis. Results The enhanced tool retained the four existing core competencies while integrating four new items assessing digital literacy to address societal demands and curricular shifts. Among 56 existing items, nine with low suitability were revised to resolve structural issues identified in factor analysis, thereby improving reliability. Test equating ensured interpretative accuracy and comparability of results pre- and post-enhancement. Longitudinal analysis revealed significant score increases in 2024 compared to the previous four years, even after adjusting for test modifications through equating. Conclusions The phased enhancement maintained structural stability while improving reliability and validity, enhancing flexibility, and meeting new demands for long-term applicability. Moreover, the established system supports sustained improvement through longitudinal evaluation and feedback, providing evidence-based insights into competency development and educational effectiveness. This study contributes to the practical utilization and future advancement of core competency diagnostic systems in higher education.