Dynamic reinforcement learning for actors

In reinforcement learning (RL), as well as in generative AI, stochastic selection is insufficient for achieving human-like flexible balancing between irregularity and rationality(or exploration and exploitation), even with a temperature parameter. Dynamic RL, proposed in this paper, brings about a major qualitative shift in RL, including exploration, from static to dynamic. It learns chaotic system dynamics that generate actions inherently containing deterministic, state-dependent exploration. Dynamic RL learns the global dynamics using a local index called sensitivity, which measures how much the input neighborhood contracts or expands into the corresponding output neighborhood through each neuron's processing. While Sensitivity Adjustment Learning (SAL) prevents excessive convergence of the dynamics, Sensitivity-controlled Reinforcement Learning (SRL) modulates them - to converge more to improve reproducibility around better state transitions with positive TD error, and to diverge more to enhance exploration around worse transitions with negative TD error. Here, Dynamic RL is applied only to the actor in an Actor-Critic framework. It was tested on two dynamic tasks, functioning effectively without external exploration noise or backward computation through time. Dynamic RL demonstrated excellent adaptability to unfamiliar situations with chaotic dynamics flexibly controlled, although some issues remain. Drawing parallels and contrasts between exploration and thinking, it is hypothesized that explorationgrows intothinkingthrough learningbut autonomously resumes in ongoing adverse situations. A mechanism to achieve this process using Dynamic RL is also suggested. Finally, despite being presumptuous, the author urges researchers to stop advancing this research due to its potentially fatal risks, aiming to encourage discussion.