The Application of an Improved UCB Algorithm Based on Deep Reinforcement Learning in the Game of Surakarta

Monte Carlo Tree Search (MCTS) is a widely used search algorithm in game theory. Its core idea is to estimate the value of each decision node by simulating random games. The Upper Confidence Bound (UCB) algorithm, as a key node selection strategy within MCTS, balances the exploration-exploitation trade-off. To further enhance the performance of MCTS, this paper proposes an improved approach that utilizes the prior probability <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$P$</tex> generated by a policy neural network to guide node selection and effectively prune the search tree, thereby improving search efficiency. For the complex game of Surakarta, the paper introduces a control parameter <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$A$</tex> to adjust the exploration aspect. Additionally, by incorporating a Deep Q Network (DQN), both <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$P$</tex> and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$A$</tex> are optimized through policy adjustment, with the agent continuously interacting with the environment to fine-tune the weight coefficient w and maximize the balanced guidance resulting from the weighted combination. The improved MCTS significantly enhances the decision-making efficiency and game performance of Surakarta, especially in scenarios with numerous pieces. This approach provides a novel perspective for optimizing MCTS.