Hyperparameter Adaptive Adjustment Method Based on Self-Attention Mechanism

Aiming at the problem that it is difficult to adjust the hyperparameters dynamically when the meta-heuristic algorithm is running, this paper proposes a hyperparameter adaptive adjustment method based on the attention mechanism. Through a refined design of four core elements of reinforcement learning, the hyperparameter adaptive adjustment strategy can be trained once and inferred everywhere. Simultaneously, to strengthen the decision-making performance of the agent, a Transformer-based deformation network is finely designed. The experimental evaluation shows that the hyperparameter adaptive adjustment method based on the attention mechanism helps the metaheuristic algorithm to find a better solution and has a better performance in the robustness test, which proves that the suggested approach can adequately alleviate the issue of hyperparameter adaptive adjustment.