Guessing Without Momentum: A Language Model States the Rules but Skips the Test

A language-model agent can reach a goal in an unfamiliar environment while its account of the rules remains inadequate for later levels. We study this gap with a quantised Qwen3.8-Flash-Next model in ARC-AGI-3, an interactive benchmark whose rules and goals must be discovered through play. The philosopher Charles Sanders Peirce described inquiry as a cycle of proposing explanations, deriving predictions, and testing them against experience. We use these stages to analyse recorded plays, changes to what the model observes and which tools it has, and preliminary fine-tuning. To test a proposed fix, we restore a game and the model’s conversation at a chosen decision point and let the model continue repeatedly, with and without the fix, under the same token budget. At four points from failed plays, 19 of 40 continuations without any fix still completed the level, so a fix must be compared with this unaided recovery. Computed summaries of the game added to every turn coincided with fewer completed levels. Factual notes improved the next response, but a broader correction package produced no detectable gain in score. A cue to an intermediate board the model had not been shown coincided with higher completion at selected points in one game, with no detected benefit at three others. Fine-tuning made the model check its rules far more often and write longer replies, but it completed fewer held-out levels under the same time limits. More checking did not mean better play, which raises the question of when a check is worth its cost. Human and model traces motivate momentum in inquiry: continuing to investigate an unresolved relation even when the goal can already be reached. We propose a drive to understand as an explanation of the human pattern. Whether such inquiry improves rule revision and transfer remains to be tested.