Development of a Numeracy Assessment Instrument for Vocational High School (VHS) Students

Numeracy skills are essential components that must be mastered by Vocational High School (VHS) graduates to enter the workforce. Developing a quality instrument based on the Item Response Theory (IRT) approach is necessary to measure numeracy skills. Since the development process involved VHS students, the instrument is developed based on the characteristics of the users. This study aims to (1) develop a quality numeracy assessment instrument for VHS students and (2) describe the numeracy skill profile of VHS eleven graders in Sleman Regency based on the assessment result. The subjects of the wide-scale trial were 374 students from nine VHSs in the Sleman Regency of Indonesia, assessed in nine study areas. Data analysis used the Item Response Theory (IRT) of the Quest program with the outputs: (1) An instrument consisting of 35 questions involving multiple-choice, complex multiple-choice, short answer, and essay. All items are valid qualitatively based on expert judgment and quantitatively based on infit and outfit MNSQ value and unidimensionality. The test item reliability value is 0.96, including in the special category, and the person reliability estimateis 0.89, categorized as good. Based on Total Information Function (TIF) and Standard Error of Measurement (SEM), the instrument is reliable in estimating the numeracy skills of the students with the ability ranging from -3.2 to 3.3 logit or covering the ability categories of low, medium, and high. The difficulty level of all items is in the range of -2.0 to 2.0, falling into the good category. (2) Most students, covering 67% of VHS students in Sleman Regency, have numeracy skills at the Basic level.