Consistent Responses on Concept Items Reflect Confidence, and Vice Versa

Abstract In educational research and practice, conducting concept tests is crucial for designing effective learning environments. Various ways have emerged to enhance concept items by adding a second tier and assessing additional variables, sparking an ongoing debate on their interplay and the implications for test development. This study compares two of those variables that are widely used across numerous research fields: On the one hand, confidence ratings have been added to test items, where learners self-assess their certainty and wrong answers that are given confidently are seen as indicative of conceptual misunderstandings. On the other hand, isomorphic prompts have been employed, which present pairs of structurally similar problems, highlighting the presence of possible misconceptions whenever learners consistently select wrong answer options, i.e. making the same mistake in different scenarios. So far, the measurement of confidence and consistency has coexisted in the literature, with minimal effort to contrast both approaches. The purpose of this study is to shed light on whether there is a methodological redundancy or whether the approaches differ significantly. Specifically, we investigated whether confident students are more likely to be consistent in their answers to isomorphic prompts, and conversely, whether students who answered the isomorphic prompts in a more consistent manner are also more likely to be confident. To this end, we operationalised confidence as well as consistency and analysed their relationship through the lens of an abstract algebra test instrument administered to N = 112 participants, comprised of undergraduate mathematics students and pre-service mathematics teachers. Our results show a strong and statistically significant correlation ( ρ = 0 . 76 , p < 0 . 001) between the two variables. Further analysis using a Mann–Whitney-U-test reveals that participants who tend to answer consistently on isomorphic prompts also tend to be more confident ( r = 0 . 78 , p < 0 . 001), and vice versa ( r = 0 . 73 , p < 0 . 001), suggesting that consistency and confidence may capture overlapping constructs, thus introducing methodological redundancy. Implications for test development, assessment methods of misconceptions and future research are discussed.