Dynamic Stochastic Reorientation Particle Swarm Optimization for Adaptive Latent Factor Analysis in High-Dimensional Sparse Matrices

The latent factor analysis (LFA) model has been widely used to uncover latent relationships from high-dimensional sparse (HiDS) matrices. However, the performance of LFA depends largely on the hyper-parameter value used in the model training. Traditional hyper-parameter tuning methods such as grid search suffer from inefficiency and inaccuracy. In recent years, the particle swarm optimization (PSO) algorithm offers an intelligent approach to adaptively adjust the hyper-parameter of LFA. However, the global optimal solution of the hyper-parameter tuning problem is not fixed due to its dynamic decision space. Therefore, it is difficult for PSO to determine the best hyper-parameter for each training iteration. To address this problem, this paper proposes a novel hyper-parameter adaptive adjustment algorithm called dynamic stochastic reorientation PSO (DSR-PSO) that adapts to constantly changing decision spaces. By randomly adjusting the search directions of particles and perturbing the elite particles, the dynamic property of the DSR-PSO can be enhanced, so that the hyper-parameter can be adjusted in real time throughout the model training process. Furthermore, this paper proves the convergence of the DSR-PSO and gives its convergence condition by discussing the distribution of the characteristic roots. Finally, this paper proposes the DSR-PSO-based LFA (DPL) model by incorporating the DSR-PSO-based hyper-parameter adjustment into the LFA to promote its model training, and analyzes its complexity. Experimental results on benchmark datasets show that the proposed DPL surpasses state-of-the-art LFA models in terms of accuracy and efficiency.