An Empirical Study of Task Progress Abstraction for Count-Based Exploration in MiniGrid DoorKey

Count-based exploration depends on which experiences share a visitation count. This study examines a hand-specified Task Progress Count Bonus (TPCB) that combines position with three current stages in MiniGrid DoorKey: a locked door without the key carried, a locked door with the key carried, and an unlocked door. Six conditions use identical symbolic policy inputs. Following development and random-trajectory bonus calibration, 60 confirmation runs cover two grid sizes and five new training seeds, with 100,096 interactions per run. Each model is evaluated on 100 reserved maps under stochastic and deterministic execution. TPCB reaches 100% mean stochastic success in DoorKey6. In DoorKey8, its 67.8% success compares with 66.2% for a compact-context control; the paired difference is 1.6 percentage points, with a 95% bootstrap interval of -31.2 to 42.6 points. A subsequent reward-schedule study uses 27 development runs and 15 fresh-seed validation runs. The selected TPCB setting reaches 96.0% during development but 49.4% in validation, versus 85.0% for the selected compact-context setting. This follow-up compares complete selected configurations, not count representations alone. TPCB's deterministic success in the original DoorKey8 confirmation is only 2.4%; a separate development diagnostic verifies cycles in all 179 deterministic failures examined. The results support task-stage counting in the smaller setting, but neither a consistent advantage over informative controls nor reliable deterministic execution. Privileged inputs, few training seeds and fixed map panels limit generalisation.