An Improved Adam Algorithm Based on Multi-strategy

Aiming at the problems of slow convergence, easy to miss the global optimal solution and poor effect in dealing with high-dimensional vectors, an improved Adam algorithm combining adaptive parameters and composite gradient is proposed. Firstly, adaptive parameters are used to calculate the difference between the first-order momentum and the current gradient, and correct the search direction, so as to improve the gradient deviation caused by outliers and help improve the convergence speed. Secondly, the current gradient and the first-order momentum are fused to form a composite gradient, so as to determine a more accurate search direction and improve the global optimization ability. Finally, the random block coordinate method is used to determine the gradient update mode, which greatly reduces the memory and CPU utilization. Experimental results show that, compared with other five methods, the proposed algorithm has the fastest convergence speed, higher accuracy, less computational overhead, and the optimal performance.