State-Self-Determining Reinforcement Learning for Adapting to Unknown Environments and Elucidating Higher Brain Functions

The real world is an indefinite environment in which the probability space is not specified in advance. Here, we developed a learning model in which the state space is expanded so that it refers to the arbitrary length of previous states, based on two criteria: experience saturation and decision uniqueness of action selection. The model was tested by behavioral tasks called a two-target search task and action sequence task that have contributed to elucidating of functional roles of higher motor areas in the cerebral cortex. The proposed state-self-determining reinforcement learning model serves as a basis not only for high adaptability to an indefinite environment but also for building a computational theory of higher brain functions.