Hydraulic-Supports Alignment by TD3 with Segmented Experience Pool

Abstract Hydraulic-supports alignment is to keep the coal mining face in line and is heavily influenced by the various geological states. The experiences produced by the moving process are unbalanced, which leads to the agent not learning important knowledge from the rare samples. This paper is the first to introduce the reinforcement learning to the hydraulic-supports alignment, and establish the Markov optimal decision model by TD3 algorithm. Aiming at the imbalance issue of the experience, this paper proposes a segmented experience pool and three sampling replay mechanisms according to the characteristics of the moving process with various geological states. Experimental results show that the improved TD3, utilizing a segmented experience pool with three different replay mechanisms, could effectively identify the optimal moving policy and achieve significant convergence in cases involving both normal movement and insufficient movement of hydraulic-supports. In contrast, the TD3 performs inadequately and struggles to find the optimal policy.