Dynamics and structure of generalization during reinforcement learning in human brains and artificial networks

Goal-directed decision making amidst an overwhelming stream of sensory input requires learning internal representations that capture a task's underlying structure.Importantly, such internal abstractions enable generalization.Representing an object's shape but ignoring its color, for instance, means that anything learned about a green triangle will generalize to red triangles.Here, we investigate this dynamic interaction of task representation learning and generalization.Human participants and artificial neural networks were trained with the same contextual reinforcement learning task.Analyses of human data reveal that participants learned an abstract task structure, which becomes detectable in the orbitofrontal cortex (OFC) after learning.Recurrent neural networks trained on the same learning curriculum exhibit similar abstractions of task representations over time.Notably, we find that the similarity structure of the networks' internal task representations affects how weight updates after a single example alter network behavior and representations on other trials.The network's progressing context differentiation in its internal layers hence leads to generalization of single experiences to other events within the same context.Ongoing work aims to gain a mechanistic understanding of model observations and contrast them with learning dynamics in the human brain.