The Role of Reinforcement Learning in Pragmatic Reasoning Tasks: Modeling and Validating the Sources of Individual Differences

Abstract In Gricean pragmatics, inference during communication is regarded as a form of rational, domain-general reasoning about the intentions of other agents. Studies using the pictorial communication “reference game” task are sometimes used in support of this hypothesis. Yet, measures of pragmatic reasoning in this task sometimes reveal poor performance, with participants requiring many rounds of play before they exhibit patterns which match Gricean inferences, and demonstrating substantial individual differences in behavior. Do these results challenge the idea of widespread inferencing via fundamental social competence? We advance an alternative proposal here, which posits that these patterns emerge as a factor of the way participants perform pragmatic reasoning in a task: namely, they prefer to use simpler interpretation strategies until experience motivates the use of additional resources. Building off of work modeling task adaptation as reinforcement learning, we use the cognitive architecture ACT-R to simulate the expected behavior of individuals with this kind of resource-rational performance algorithm, subject to individualized parameters for reinforcement learning. These simulations provide a proof-of-concept for our adaptation proposal, recreating known patterns and generating new concrete predictions for the particular domain-general sources of individual variance in reference game tasks. We then go on to validate some of these new predictions in a pre-registered experiment, and find that pragmatic response behavior is indeed related to a participant’s general persistence in self-directed exploration of strategies for task completion. Our results offer a path to reconcile variable empirical data with models of core pragmatic competence. From a broader perspective, we see this as an important step towards more robust theories of performance factors in pragmatic reasoning, and ultimately, a case study in the value of process-level computational modeling.