Investigating the Matching Law in Transformer-Based Reinforcement Learning Agents

This study examines whether a Transformer-based reinforcement learning agent follows the Matching Law—the principle that organisms allocate responses in proportion to rewards. We trained a Transformer agent on a two-choice sequential decision task with varying reward contingencies to see if its choice behavior matches relative reward rates. The Transformer’s attention-based architecture was leveraged to learn temporal patterns and adapt its policy. We found that the agent’s choice proportions closely mirrored the reward ratios across conditions, demonstrating adherence to the Matching Law. Log-log analyses showed an approximately linear relationship between response and reward ratios. These results suggest that even advanced AI agents like Transformers can spontaneously exhibit Matching Law behavior, offering insights into links between artificial and biological learning.