A Validation Study of the Higher-order Thinking Competency Diagnostic Tool through Item Analysis

The purpose of this study is to validate a diagnostic tool for assessing university students' Higher-order thinking competency through item analysis and to provide rational and valid evidence for its application in teaching and learning support. The final analysis was conducted using responses from 261 students in the natural sciences and engineering disciplines at A University. The results are as follows: First, basic statistical analyses confirmed that the model fit was appropriate. Second, item characteristics were examined using Classical Test Theory(CTT) and the Two-Parameter Item Response Theory(2-PL IRT), ensuring validity in terms of difficulty and discrimination. Third, the overall item analysis indicated that the Higher-order thinking competency diagnostic items aligned with the levels of difficulty established during the initial development phase. To effectively diagnose students' competencies, it is essential to develop diagnostic tools that are objective and valid, which can serve as a rational basis for non-curricular educational support aimed at enhancing foundational learning competencies. Unlike previous studies that were limited to basic statistical analysis or the development of self-reported items, this study is significant in developing and validating a high-level, objective diagnostic tool for Higher-order thinking competency. The findings of this competency diagnostic tool are expected to serve as fundamental data for establishing and implementing non-curricular educational support systems.