Behavioral Probes for Identifying Proxy-Directed Behavior in Simple Gridworld: A Preliminary Empirical Study
This preliminary empirical study investigates behavioral differences between intended-goal-reward and proxy-reward reinforcement learning agents in a simple gridworld environment. Two tabular Q-learning agents are trained in a 5 × 5 gridworld using different reward objectives and evaluated across five random seeds and five intended-goal locations. Evaluation measures include intended-goal success rate, cumulative return, proxy-region entry rate, and two behavioral indicators (PRI-A and PRI-Q). The results show contrasting reward-conditioned behaviors and demonstrate how the implemented behavioral indicators characterize action preferences in this controlled setting. However, the study does not establish goal misgeneralization following a distribution shift or demonstrate that the behavioral probes outperform conventional performance metrics. This work is presented as a preliminary empirical study.
Authors
- Maithili Jadhao
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-13
- DOI
- https://doi.org/10.5281/zenodo.24187572
- Primary Topic
- Reinforcement Learning in Robotics
- Type
- preprint