Evaluation of the Progress Test of a Medical school according to the SOLO Taxonomy theoretical framework

ABSTRACT Introduction: the training of future graduates from medical schools for responsible and qualified health care practice is a significant challenge. The goal is for them to be equipped to solve problems that require higher-order cognitive skills. Thus, evaluating the acquisition of such competencies becomes crucial. One assessment method that has been gaining attention in medical education is the Progress Test (PT). Cognitive theories have advanced educational research related to assessment processes. In our study, we used the Structure of Observing Learning Outcome (SOLO) taxonomy to evaluate and categorize the items of the PT applied at a medical school. The SOLO taxonomy (ST) allows for the necessary cognitive analysis required for performing specific tasks, enabling a comprehensive observation of the student’s understanding. We also applied the Classical Test Theory (CTT) in our study, calculating the difficulty index (DFI) and discrimination index (DI) for each multiple-choice item (MCI) of the PT and correlating them with the SOLO classification. Objective: the objective of this study is to evaluate the characteristics of the PT applied in a private medical school, analyzing its items based on the assumptions of ST and correlating them with CTT. Materials and Methods: this is a descriptive study with a quantitative and qualitative approach. According to the principles of ST, we conducted the analysis and characterization of the items from a PT applied in a private medical school and correlated them with the DFI and DI. Results: we found a balance between surface learning (SL) and deep learning (DL) across the total items, as well as a direct relationship between the levels of DL and MCIs consisting of clinical cases. We did not find statistically significant differences between the SOLO categories regarding the means of DFI and DI. Conclusion: t he analysis of assessment activities should not be restricted to psychometric properties. Taxonomic tools, such as the ST, can significantly aid in conducting these activities, aligning assessments with the curriculum and facilitating the creation of tests appropriate for the desired level of learning, thereby promoting effective teaching progression.