Validity evidence based on response processes for an instrument assessing functional thinking

Abstract Various studies from the perspective of early algebra have employed written tests to assess students’ functional thinking. However, there has been little research on the validity of the interpretations derived from these instruments. The purpose of this study was to validate the scores of an instrument designed to assess functional thinking in third-grade students by providing evidence based on response processes. A test and a retrospective cognitive interview were administered to 24 third-grade students from a school in Santiago, Chile. Responses from both the test and the interview were coded and compared to assess the agreement between the interpretations derived from each instrument. The results showed a strong and statistically significant correlation in overall performance and revealed that the item with the greatest divergence between the test and the interview involved describing the relationship between variables using natural language. In conclusion, the instrument offers a reliable means of capturing students’ functional thinking, although it may underestimate their performance when expressing functional rules using natural language. Future research could use this instrument while incorporating brief interviews for students in the numerical-functional category, in order to more accurately capture their ability to generalize in natural language without relying on their writing skills.