Individual Differences in Problem-Solving Behavior in Reasoning Tests: The Role of Item Difficulty and Item Position

Matrix reasoning performance is often treated as a static indicator of cognitive ability, although it reflects a dynamic process that unfolds across items. The present study examined how reasoning ability and visuospatial working memory (vsWM) interact with item difficulty and item position to shape problem-solving behavior. Eye movements were recorded from 300 participants during a figural matrices test. Three indices of problem-solving behavior were analyzed using Bayesian multilevel models: toggle rate (ToR), proportional time on matrix (PoM), and proportional time to first fixation on response alternatives (PoFA). Results showed that higher reasoning was consistently associated with behavior indicative of constructive matching, reflected in lower ToR, higher PoM, and higher PoFA. Further, individuals with higher reasoning ability showed stronger adaptations in problem-solving behavior in response to changing task demands. Evidence regarding vsWM was more nuanced, revealing selective interactions with reasoning ability and test characteristics rather than a consistent main effect across measures. Item position revealed substantial between-person variability, which exceeded the variability associated with item difficulty. This suggests that individuals differ more strongly in how their behavior evolves across the test than in their responses to increasing difficulty. Overall, the findings highlight that problem-solving behavior in reasoning tests reflects a dynamic interplay between individual characteristics and test characteristics, emphasizing the importance of modeling both simultaneously when investigating reasoning performance.