“PyTDL”: A versatile temporal difference learning algorithm to simulate behavior process of decision making and cognitive learning

Humans and animals excel at learning complex tasks through reward-based feedback, dynamically adjusting value expectations and choices based on past experiences to optimize outcomes. However, understanding the hidden cognitive components driving these behaviors remains challenging. Neuroscientists use the Temporal Difference (TD) learning model to estimate cognitive elements like value representation and prediction error during learning and decision-making processes. However, traditional TD algorithms fall short in diverse and dynamic tasks due to their fixed patterns. We present PyTDL, a Python-based modular framework that enables customizable value updating functions and decision policies, effectively simulating dynamic, non-linear cognitive processes. PyTDL's utility was demonstrated by modeling the decision-making processes of animals in two cognitive tasks under uncertain conditions. As open-source software, PyTDL offers a user-friendly GUI and APIs, empowering researchers to tailor models for specific tasks, align computational models with empirical data, and advance the understanding of brain learning and decision-making in complex environments.