Perturbed Gradient Methods for Data Classification via Regularized Extreme Learning Machine

ABSTRACT Extreme Learning Machines (ELMs) are efficient single‐hidden‐layer neural networks for classification tasks, yet optimizing their output weights using the Moore–Penrose pseudoinverse can be computationally expensive and numerically unstable, particularly for large datasets or datasets leading to an ill‐conditioned problem. To overcome these limitations, this paper proposes a perturbed adaptive gradient descent algorithm to solve an unconstrained optimization problem, in particular, a regularized least squares formulation of the ELM training problem. The method incorporates a perturbation and self‐adaptive stepsize technique to enhance convergence speed and stability, building on gradient descent techniques. We provide a theoretical convergence analysis and empirically evaluate the algorithm along with two other adaptive gradient descent methods on three UCI datasets, Breast Cancer Wisconsin, Glass Identification, and Jute Pest, using various activation functions. Experimental results demonstrate that the proposed approach achieves competitive classification accuracy and fast convergence, especially for larger network sizes. This work advances the efficiency and robustness of ELM training, offering alternative methods for data classification.