Reinforcement Learning Timing Optimization for Safe Robot Navigation in Dynamic Environments

Reinforcement learning theoretically needs to keep re-learning new optimal policies for autonomous robot navigation when the environment is dynamic. Given no reinforcement learning is failure-free, however, the practical deployment of reinforcement learning raises a new but fundamental question: when to re-learn and when not to be safer in the long-term robot navigation? An early re-learning could encounter more failures due to unnecessary and unsafe explorations in the learning process while a late re-learning cannot avoid failures caused by the deteriorating policy. To determine the optimal timing of learning with minimal failures, we propose a Value of Learning approach that maximizes the difference between the learning gain and the learning cost. The learning gain estimates the performance improvement with respect to time and the learning cost evaluates the learning-induced failure risk, which is assumed to relate to environment changes and exploration strategies. A dynamic maze experiment validates that our approach can find an optimal re-learning timing with significantly fewer failures if compared with “always learning” and “no learning”, which are two representative cases of the early and late learning.