An Improved Imitation Learning Algorithm Based on Markov Chain

An improved imitation learning algorithm based on Markov chain is proposed, aiming to enhance the performance of autonomous driving in complex environments. Imitation learning learns task execution by observing expert behavior, while Markov chains are used to model state transition processes. The existing imitation learning algorithms have limited ability to handle complex tasks, especially in natural environments. The algorithm proposed in this article improves learning efficiency and autonomous control performance by introducing strategy iteration and reward function design for sudden obstacles. The core idea is to introduce Markov decision processes into traditional imitation learning frameworks to improve learning efficiency and performance. By acquiring a refined strategy, the objective is to mimic the agent's actions within the environment to closely resemble those of the expert. To achieve this, first train an initial strategy using expert examples. Then, use this strategy to generate new examples and update the strategy using these examples. This process iterates continuously until the strategy converges. The experimental results show that on the lunar environment autonomous driving datasets, the accuracy of our algorithm reaches 95% with a mean square error of 0.02, while the accuracy of the comparative algorithm is the highest at 90% with a mean square error of 0.05. This indicates that the improved algorithm can more accurately learn and imitate the driving behavior of experts. In addition, the performance of the algorithm is stable at different iteration times, indicating good convergence.