A Modified U-Tree Algorithm Based on Sarsa(λ)

Using Traditional U-Tree algorithm the agent will do a value iteration for each step in the environment, that is, one-step dynamic programming of the Q value, and update the current Q value with the estimation state at the next time, regardless of the future. This article we propose a modified U-Tree algorithm based on Sar-sa (λ) algorithm, and use the Sarsa (λ) algorithm to update the Q value, which is called SU-Tree algorithm. When the Agent arrives at the agent target state or the penalty state, the Q value of the whole path is updated from the starting point to improve the convergence speed of the algorithm. By simulating the new algorithm in the 4×3 grid issues, agent can find the non-shock path from the beginning to the end more quickly than the original U- Tree algorithm.