Activation Control of Laguerre Neural Network

The Laguerre Neural Network (LaNN) employs Laguerre polynomials (LPs) as the activation functions of its hidden neurons (HNs) to approximate the nonlinear functions (NFs). From the perspective of statistics, the more LPs are activated, the more LP-based activation functions (LPAFs) of the HNs will operate, and thus the smaller approximation error (APE) will be suffered by the LaNN, while leading to more energy consumption since activating any LP has to consume energy. Therefore, how to control the activation of LPs to make the optimal tradeoff between approximation accuracy and energy consumption arises as an interesting issue. To address this issue, this paper first establishes an energy-constraint probability-based approximation-accuracy optimization-theoretical (EPAO) framework, considering the probability of obtaining the minimum APE (MAPE) as the objective while the energy consumption of the LaNN as the constraint, then reveals the optimal number of HNs and thus their optimal LPAFs, and finally discloses the optimal activation probability for each LP. Based on our discovered theoretical results, a novel algorithm for LaNN to maximize the probability of obtaining the MAPE at the acceptable level of energy consumption is proposed. Theoretical analysis verifies its performance, and field experiments verify that, compared with existing methods, it improves the probability of obtaining the MAPE by 6.00% to 20.26% at the similar cost of energy. To our best knowledge, this is the initial work for LaNN towards joint optimization of its energy efficiency and approximation accuracy.