LLM-Augmented Deep Reinforcement Learning for Complex Task Planning

Complex task planning is crucial for enhancing the production autonomy and efficiency. The current Deep Reinforcement Learning (DRL)-based complex task planning algorithms struggle to converge to the optimal policy during training due to a lack of understanding of the environment and strategic knowledge. To counteract this challenge, we introduce a Large Language Model (LLM)-augmented DRL algorithm named Monte Carlo Tree Search Algorithm Assisted by a Large Language Model with Policy and Value Networks (LPV-MCTS). LPV-MCTS leverages the preexisting knowledge of LLM to accelerate the initial training phase of DRL, enabling it to assimilate high-quality data from the outset, thereby hastening the algorithm's environmental comprehension and the convergence of the policy. We then compare the proposed LPV-MCTS algorithm with other complex task planning algorithms. The empirical results demonstrate that the LPV-MCTS outperforms the baselines in terms of planning efficiency, generalizability, and training time efficiency.