A Mathematical Analysis of Effective Learning and Decision-Making in Neuronal Circuits

Learning and decision-making are fundamental aspects of cognition, but much remains unknown about how they are accomplished by the brain. We investigate these topics through the framework of effective task performance. We first study action selection on forced choice tasks, in which it is generally assumed that agents attempt to maximize reward rate. We compare the standard formulation of reward rate, expected reward divided by expected time or E[R]/E[T], to an alternative formulation, the expectation of reward divided by time or E[R/T], where R denotes reward size and T denotes decision time. Both theoretical and empirical results suggest that E[R/T] may in many cases better describe behavior. We derive a formula for E[R/T] in the context of drift-diffusion models and find parameter regimes where it differs from the classical formula. These results may provide a new lens through which to analyze experimental data related to decision-making. Next, we investigate how the learning mechanisms present in the basal ganglia impact task performance in a variety of settings. Corticostriatal synapses, which serve as the primary input to the basal ganglia, undergo spike-timing-dependent plasticity (STDP) modulated by dopamine. We introduce three forms of dopamine-modulated STDP and analyze, both mathematically and with simulations, their performance in several biologically relevant scenarios. We find that each plasticity rule is well-suited to some of the scenarios studied but falls short in others. This demonstrates that different tasks require different forms of synaptic plasticity, and suggests that regions of the striatum -- or other brain areas impacted by dopamine -- with distinct computational functions may show variations in the STDP mechanisms they implement.